{"id":37739369,"url":"https://github.com/pavelacamposp/data-driven-quad-control","last_synced_at":"2026-01-18T08:46:00.727Z","repository":{"id":301328013,"uuid":"909175829","full_name":"pavelacamposp/data-driven-quad-control","owner":"pavelacamposp","description":"Data-driven position control for quadcopters in Genesis: Reinforcement Learning and Data-Driven MPC","archived":false,"fork":false,"pushed_at":"2025-07-16T22:40:14.000Z","size":9571,"stargazers_count":7,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-17T00:14:12.409Z","etag":null,"topics":["control-systems","data-driven","data-driven-mpc","drone","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pavelacamposp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-27T23:48:47.000Z","updated_at":"2025-12-06T06:31:53.000Z","dependencies_parsed_at":"2025-06-30T11:39:11.276Z","dependency_job_id":"c76b1a49-1402-436f-9252-39ca59c0ff1b","html_url":"https://github.com/pavelacamposp/data-driven-quad-control","commit_stats":null,"previous_names":["pavelacamposp/data_driven_quad_control","pavelacamposp/data-driven-quad-control"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pavelacamposp/data-driven-quad-control","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pavelacamposp%2Fdata-driven-quad-control","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pavelacamposp%2Fdata-driven-quad-control/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pavelacamposp%2Fdata-driven-quad-control/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pavelacamposp%2Fdata-driven-quad-control/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pavelacamposp","download_url":"https://codeload.github.com/pavelacamposp/data-driven-quad-control/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pavelacamposp%2Fdata-driven-quad-control/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28534148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["control-systems","data-driven","data-driven-mpc","drone","reinforcement-learning"],"created_at":"2026-01-16T14:05:40.828Z","updated_at":"2026-01-18T08:46:00.718Z","avatar_url":"https://github.com/pavelacamposp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data-Driven Position Control for Quadcopters: Reinforcement Learning and Data-Driven MPC\n\n\u003cdiv align=\"center\"\u003e\n\n[![GitHub Actions status](https://github.com/pavelacamposp/data-driven-quad-control/actions/workflows/ci_workflow.yml/badge.svg)](https://github.com/pavelacamposp/data-driven-quad-control/actions/workflows/ci_workflow.yml)\n[![Codecov](https://codecov.io/gh/pavelacamposp/data-driven-quad-control/graph/badge.svg)](https://codecov.io/gh/pavelacamposp/data-driven-quad-control)\n[![Ruff](https://img.shields.io/badge/Lint%20%26%20Format-Ruff-blue?logo=ruff\u0026logoColor=white)](https://github.com/astral-sh/ruff)\n[![Python](https://img.shields.io/badge/Python-3.10%20|%203.12-blue)](https://docs.python.org/3.10)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n\u003c/div\u003e\n\n![Data-driven Controller Comparison Animation](docs/resources/data_driven_quad_control_sim.gif)\n\nThis project implements data-driven controllers for position control of quadcopters in [Genesis](https://github.com/Genesis-Embodied-AI/Genesis), based on **Reinforcement Learning** and **nonlinear data-driven Model Predictive Control (MPC)**.\n\nBuilding on the `HoverEnv` drone environment from the [Genesis repository](https://github.com/Genesis-Embodied-AI/Genesis/blob/main/examples/drone/hover_env.py), which uses rotor RPMs as actions, we implement a high-level **Collective Thrust and Body Rates (CTBR) controller** to enable the use of CTBR actions. These actions consist of the drone’s total thrust and angular velocities about the roll, pitch, and yaw axes.\n\nBy using the CTBR controller as a low-level controller, the drone system is simplified, enabling the use of direct data-driven controllers for position control, such as the data-driven MPC controller designed in this project.\n\nAdditionally, a **PID tracking controller** based on an SE(3) controller is provided. This controller is used for drone stabilization during data collection and as a baseline for performance comparison.\n\nThe data-driven MPC control system builds on the implementation from [direct-data-driven-mpc](https://github.com/pavelacamposp/direct-data-driven-mpc).\n\n## Key Features\n- **Drone Controllers:**\n    - Collective Thrust and Body Rates (CTBR) controller\n    - PID tracking controller (SE(3)-based)\n    - Nonlinear data-driven MPC controller\n    - Reinforcement Learning controller\n- **Vectorized drone environment** with support for various action types: rotor RPMs, CTBR, and CTBR with fixed yaw rate.\n- **Nonlinear data-driven MPC integration:**\n    - Parallel grid search for controller parameter identification\n    - Evaluation system for data-driven MPC controllers\n- **Reinforcement Learning:** Customizable training and evaluation scripts.\n- **Data-driven controller comparison:** Parallel evaluation of controllers.\n\n## Table of Contents\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [Collective Thrust and Body Rates (CTBR) Controller](#collective-thrust-and-body-rates-ctbr-controller)\n  - [Usage Example](#usage-example)\n- [PID Tracking Controller](#pid-tracking-controller)\n  - [Usage Example](#usage-example-1)\n- [Reinforcement Learning Controller](#reinforcement-learning-controller)\n  - [Training a Policy](#training-a-policy)\n  - [Evaluating a Policy](#evaluating-a-policy)\n- [Nonlinear Data-Driven MPC Controller](#nonlinear-data-driven-mpc-controller)\n  - [DD-MPC Parameter Grid Search](#dd-mpc-parameter-grid-search)\n  - [DD-MPC Controller Evaluation](#dd-mpc-controller-evaluation)\n- [Controller Comparison](#controller-comparison)\n  - [Controller Comparison System](#controller-comparison-system)\n  - [Control Trajectory Plotting](#control-trajectory-plotting)\n- [License](#license)\n\n## Requirements\n- **Python 3.10** or later.\n- **CUDA-capable system** required to run the Genesis simulator.\n\n\u003e [!IMPORTANT]\n\u003e On **Windows**, you may need to enable [long path support](https://pip.pypa.io/warnings/enable-long-paths) to avoid Git installation errors due to path lengths. Alternatively, you can run the following command to enable long path support directly in Git:\n\u003e ```bash\n\u003e git config --global core.longpaths true\n\u003e ```\n\n\u003e [!IMPORTANT]\n\u003e This project was developed and tested on Unix (Ubuntu).  \n**Windows is currently partially supported**: Most features work, but scripts that use PyTorch multiprocessing (Data-Driven MPC Grid Search and Controller Comparison) do not run correctly on Windows.\n\n## Installation\nFollow these steps to create a virtual environment and install the project:\n\n1. Clone the repository and navigate to the project directory:\n    ```bash\n    git clone https://github.com/pavelacamposp/data-driven-quad-control.git \u0026\u0026 cd data-driven-quad-control\n    ```\n2. Create and activate a virtual environment:\n    - Unix/macOS:\n        ```bash\n        python3 -m venv .venv \u0026\u0026 source .venv/bin/activate\n        ```\n    - Windows:\n        ```cmd\n        python -m venv venv \u0026\u0026 venv\\Scripts\\activate\n        ```\n3. Upgrade pip:\n    ```bash\n    python -m pip install --upgrade pip\n    ```\n4. Install **PyTorch with CUDA support** by following the [official instructions](https://pytorch.org/get-started/locally/), or by running the following command:\n    ```bash\n    pip install torch --index-url https://download.pytorch.org/whl/cu128\n    ```\n    **Note:** Replace `cu128` with a CUDA version supported by your NVIDIA driver.\n\n5. Install the project:\n    ```bash\n    pip install -e .\n    ```\n\n## Collective Thrust and Body Rates (CTBR) Controller\nA Collective Thrust and Body Rates (CTBR) controller converts high-level control commands (total thrust and angular velocities) into individual rotor RPMs, serving as an abstraction that simplifies the drone control system.\n\nThis controller is implemented in [`src/data_driven_quad_control/controllers/ctbr/ctbr_controller.py`](src/data_driven_quad_control/controllers/ctbr/ctbr_controller.py), and its parameters can be modified in the YAML configuration file [`configs/controllers/ctbr/ctbr_controller_params.yaml`](configs/controllers/ctbr/ctbr_controller_params.yaml).\n\n### Usage Example\nTo see the CTBR controller in operation with multiple parallel drones, run the example script with the following command:\n```bash\npython examples/drone_ctbr_controller_example.py --num_envs 5\n```\n\n## PID Tracking Controller\nThis project implements an SE(3)-based PID tracking controller that outputs CTBR commands. These commands are then followed by an external lower-level CTBR controller, following a cascade control architecture.\n\nThis controller is implemented in [`src/data_driven_quad_control/controllers/tracking/tracking_controller.py`](src/data_driven_quad_control/controllers/tracking/tracking_controller.py), and its parameters can be modified in the YAML configuration file [`configs/controllers/tracking/tracking_controller_params.yaml`](configs/controllers/tracking/tracking_controller_params.yaml).\n\n### Usage Example\nTo see the tracking controller in operation with multiple parallel drones, run the example script with the following command:\n```bash\npython examples/drone_tracking_controller_example.py --num_envs 5 --target_pos_noise\n```\n\nThe `--target_pos_noise` argument enables the addition of noise to target positions for variety.\n\n## Reinforcement Learning Controller\nThis project uses the PPO implementation from [rsl_rl](https://github.com/leggedrobotics/rsl_rl) to train agents for drone position control.\n\n### Training a Policy\nTo train a new policy with the experiment name \"drone-hovering\", 8192 parallel drone instances, CTBR actions, and 1501 iterations, run the following command:\n```bash\npython -m src.data_driven_quad_control.learning.train --exp_name drone-hovering --num_envs 8192 --action_type ctbr --max_iterations 1501\n```\n\nThe `--action_type` argument specifies the action type for the drone environment, which can be one of the following:\n- `rpms`: rotor RPMs\n- `ctbr`: CTBR commands\n- `ctbr_fixed_yaw`: CTBR commands with a fixed yaw rate of 0 rad/s\n\n#### Training Configuration\nTraining configurations can be modified in the following files:\n- [`src/data_driven_quad_control/learning/config/hover_ppo_config.py`](src/data_driven_quad_control/learning/config/hover_ppo_config.py): Defines the PPO algorithm hyperparameters.\n- [`src/data_driven_quad_control/learning/config/reward_config.py`](src/data_driven_quad_control/learning/config/reward_config.py): Defines the reward function scaling constants for each action type.\n\n### Evaluating a Policy\nTo evaluate a trained policy, run the command below. For demonstration purposes, this command uses the pretrained model included in [`models/`](models/), with the experiment directory (`--log_dir`) set to `models/demo_ctbr_fixed_yaw` and the model checkpoint index set to `1500`:\n```bash\npython -m src.data_driven_quad_control.learning.eval --log_dir models/demo_ctbr_fixed_yaw --ckpt 1500\n```\n\nTo record a video of the policy evaluation, add the `--record` argument.\n\n## Nonlinear Data-Driven MPC Controller\nThe nonlinear data-driven MPC (DD-MPC) control system for drone position control is implemented using a drone environment configured with CTBR actions with a fixed yaw angular rate of 0 rad/s. This setup simplifies the drone control system by reducing the number of control inputs to three:\n- Total thrust [N]\n- Roll angular rate [rad/s]\n- Pitch angular rate [rad/s]\n\n\u003e [!NOTE]\n\u003e The nonlinear data-driven MPC controller implementation is based on [direct-data-driven-mpc](https://github.com/pavelacamposp/direct-data-driven-mpc), which uses Python and CVXPY. This leads to slow control computations for this application due to how CVXPY builds, formulates, and solves the MPC problem. A more efficient data-driven MPC controller will be implemented in the future.\n\n### DD-MPC Parameter Grid Search\n\u003e [!WARNING]\n\u003e **Windows not yet supported:** This script does not run correctly on Windows due to multiprocessing issues with PyTorch. You can still run the [DD-MPC Controller Evaluation](#dd-mpc-controller-evaluation) script to evaluate DD-MPC controllers individually.\n\nThe DD-MPC parameter grid search system is implemented in [`src/data_driven_quad_control/data_driven_mpc/dd_mpc_param_grid_search.py`](src/data_driven_quad_control/data_driven_mpc/dd_mpc_param_grid_search.py). It identifies efficient DD-MPC parameters by spawning multiple controllers in parallel processes to evaluate unique controller parameter combinations from a defined grid. At the end of a grid search, it writes a report file summarizing the grid search results.\n\nTo run a parallel grid search with 10 parallel processes using configuration parameters defined in [`configs/data_driven_mpc/dd_mpc_grid_search_params.yaml`](configs/data_driven_mpc/dd_mpc_grid_search_params.yaml), execute the following command:\n```bash\npython -m src.data_driven_quad_control.data_driven_mpc.dd_mpc_param_grid_search --grid_search_config_path configs/data_driven_mpc/dd_mpc_grid_search_params.yaml --num_processes 10\n```\n\nTo enable visualization and debug logging during the grid search, add the `--gui` and `--debug` arguments. However, this will slow program execution.\n\n### DD-MPC Controller Evaluation\nThe DD-MPC controller evaluation system is implemented in [`src/data_driven_quad_control/data_driven_mpc/dd_mpc_controller_eval.py`](src/data_driven_quad_control/data_driven_mpc/dd_mpc_controller_eval.py). It evaluates a single controller using parameters defined in a configuration file.\n\nTo evaluate a DD-MPC controller with parameters defined in [`configs/data_driven_mpc/dd_mpc_controller_params.yaml`](configs/data_driven_mpc/dd_mpc_controller_params.yaml), run the following command:\n```bash\npython -m src.data_driven_quad_control.data_driven_mpc.dd_mpc_controller_eval --controller_config_path configs/data_driven_mpc/dd_mpc_controller_params.yaml --controller_key nonlinear_dd_mpc_approx_1_step\n```\n\n## Controller Comparison\n![Data-driven Controller Comparison Plot](docs/resources/controller_comparison_plot.png)\n*Figure: Control trajectories of tracking (PID), Reinforcement Learning (PPO), and data-driven MPC controllers.*\n\n### Controller Comparison System\n\u003e [!WARNING]\n\u003e **Windows not yet supported:** This script does not run correctly on Windows due to multiprocessing issues with PyTorch.\n\nThe controller comparison system is implemented in [`src/data_driven_quad_control/comparison/controller_comparison.py`](src/data_driven_quad_control/comparison/controller_comparison.py). It compares the following controllers in simulation for drone position control:\n- PID tracking controller (used as a baseline)\n- Reinforcement Learning controller (trained PPO policy)\n- Nonlinear data-driven MPC controller\n\nAfter each comparison run, control trajectory data collected during simulation is saved to a file in `logs/comparison/`. This data can then be used for post-evaluation and plotting.\n\nTo run the controller comparison simulation with configuration parameters defined in [`configs/comparison/controller_comparison_config.yaml`](configs/comparison/controller_comparison_config.yaml), execute the following command:\n```bash\npython -m src.data_driven_quad_control.comparison.controller_comparison --comparison_config_path configs/comparison/controller_comparison_config.yaml\n```\n\nTo record a video of the controller comparison, add the `--record` argument.\n\n### Control Trajectory Plotting\nThe script [`src/data_driven_quad_control/comparison/controller_comparison_plot.py`](src/data_driven_quad_control/comparison/controller_comparison_plot.py) is provided to plot control trajectory data generated when running the controller comparison script. It supports plotting data from multiple comparison runs, with flexible customization via the configuration files under [`configs/plots/`](configs/plots/).\n\nTo plot controller comparison trajectory data, run the following command:\n```bash\npython -m src.data_driven_quad_control.comparison.controller_comparison_plot\n```\n\n## License\nThis project is licensed under the [MIT License](LICENSE). It builds on code from the Genesis repository, which is licensed under the [Apache License 2.0](LICENSE-APACHE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpavelacamposp%2Fdata-driven-quad-control","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpavelacamposp%2Fdata-driven-quad-control","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpavelacamposp%2Fdata-driven-quad-control/lists"}