{"id":18621529,"url":"https://github.com/ethz-asl/data-driven-dynamics","last_synced_at":"2025-04-11T03:30:49.764Z","repository":{"id":38191859,"uuid":"342333815","full_name":"ethz-asl/data-driven-dynamics","owner":"ethz-asl","description":"Data Driven Dynamics Modeling for Aerial Vehicles","archived":false,"fork":false,"pushed_at":"2023-11-24T16:31:28.000Z","size":78958,"stargazers_count":103,"open_issues_count":17,"forks_count":16,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-25T08:38:07.605Z","etag":null,"topics":["drones","px4","robotics","simulation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethz-asl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-02-25T18:02:18.000Z","updated_at":"2025-03-06T03:30:44.000Z","dependencies_parsed_at":"2023-02-15T10:16:41.743Z","dependency_job_id":"23dd425c-5ee4-4a18-ba5a-1c6e0463af4e","html_url":"https://github.com/ethz-asl/data-driven-dynamics","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fdata-driven-dynamics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fdata-driven-dynamics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fdata-driven-dynamics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethz-asl%2Fdata-driven-dynamics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethz-asl","download_url":"https://codeload.github.com/ethz-asl/data-driven-dynamics/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248335352,"owners_count":21086578,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["drones","px4","robotics","simulation"],"created_at":"2024-11-07T04:12:17.701Z","updated_at":"2025-04-11T03:30:46.456Z","avatar_url":"https://github.com/ethz-asl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg align=\"right\" height=\"60\" src=\"https://user-images.githubusercontent.com/5248102/126074528-004a32b9-7911-486a-9e79-8b78e6e66fdc.png\"\u003e\n\n# data-driven-dynamics\n\n[![SysID Pipeline Test](https://github.com/ethz-asl/data-driven-dynamics/actions/workflows/sysid_test.yml/badge.svg)](https://github.com/ethz-asl/data-driven-dynamics/actions/workflows/sysid_test.yml)\n\nA dynamics model is a valuable tool for a variety of applications, including simulations and control. However, traditional methods for obtaining such models for an unmanned aerial vehicle (UAV), such as wind tunnel testing or the installation of additional sensors, can be time-consuming, inaccesible and costly. In this project, we aim to address this challeng through a data-driven dynamics modelling pipeline that uses the PX4 unified flight log (ULog) format. The pipeline can be used for obtaining a dynamics model solely by minimizing the prediciton error of the model with respect to the flight data collected by the by default on the vehicle installed sensors. The modular approach of the pipeline aims to accomodate different model structures, optimizers and various UAV airframes, including multirotors and vertical take-off and landing (VTOL) UAVs. Specifically, for the example of multirotors, the pipeline automates the joint estimation of a parametric model and the quality of the fit and integrates the former into the PX4 Gazebo flight simulator.\n\nYou can get an overview of the pipelines functionalities on the [project description website](https://galliker.tech/projects/data_driven_dynamics/) or in the following video presentation:\n\n[![Watch the video](https://img.youtube.com/vi/kAsfptZU4uk/maxresdefault.jpg)](https://www.youtube.com/watch?v=kAsfptZU4uk)\n\nMore detailed information can be found in the student paper [Data-Driven Dynamics Modelling Using Flight Logs](https://www.research-collection.ethz.ch/handle/20.500.11850/507495).\n\n## Setting up the environment\n\nThis plugin depends on [PX4 SITL](https://github.com/PX4/PX4-SITL_gazebo). Therefore custom messages of PX4 SITL needs to be linked. Therefore, prior to building this package, PX4 and corresponding SITL package needs to be built.\n\nIn case you have not cloned the firmware repository\n\n```\ngit clone --recursive https://github.com/PX4/PX4-Autopilot.git ~/src/PX4-Autopilot\n```\n\nFor internal use, our internal firmware repository is as the following.\n\n```\ngit clone --recursive https://github.com/ethz-asl/ethzasl_fw_px4.git ~/src/PX4-Autopilot\n```\n\n```\ncd \u003cFirmware Dir\u003e\nDONT_RUN=1 make px4_sitl gazebo\n```\n\nThe build directory of PX4 can be linked by setting the environment variable `PX4_ROOT` to the root of the firmware directory. Set the environment variable `export PX4_ROOT=~/src/PX4-Autopilot` or add it to your `bashrc`\n\n```\necho \"export PX4_ROOT=~/src/PX4-Autopilot\" \u003e\u003e ~/.bashrc\nsource ~/.bashrc\n```\n\nThe use the parametric model structure you need to install python 3.8 and the needed python libraries. It is strongly advised to install the pip packages in a [virtual enviroment](https://docs.python.org/3/tutorial/venv.html) setup for this project.\n\nInstall the dependencies using:\n\n```\nmake install-depdencies\n```\n\n## Build\n\nAfter the environment has been setup as described in the previous section, build the package as the following.\n\n```\nmkdir build\ncd build\ncmake ..\nmake\n```\n\n## Generating a Parametric Model from Log File\n\nLink the latest log files to your local logs folder using:\n\n```\nsource setup.bash\n```\n\nGenerate the parametric model using a log file (ulog or csv):\n\n```\nmake estimate-model [model=\u003cmodeltype\u003e] [config=\u003cconfig_file_path\u003e] [data_selection=\u003cnone|interactive|setpoint|auto\u003e] [selection_var=topic_name/variable_name] [plot=\u003cTrue/False\u003e] [log=\u003clog_file_path\u003e]\n```\n\n### Pipeline Arguments\n\n#### Model Choice\n\nThe chosen vehicle model class determines what physical effects are modelled and what parameterts need to be regressed in the system identification process.\nCurrent vehicle model choices are:\n\n- quadrotor_model (default config for quadrotor)\n- fixedwing_model (default config for cruise flight of fixed-wings and VTOLs, documented [here](https://github.com/ethz-asl/data-driven-dynamics/blob/update-documentation/doc/fixed_wing_model.md))\n\n#### Config File\n\nThe config file allows to configure the intra class vehicle variations, used log file topics, data processing and other aspects of the pipeline. The default location is in `Tools/parametric_model/configs`. The path can be passed in the make target through the `config=\u003cconfig_file_path\u003e` argument. If no config is specified the default model config is used.\n\n#### Log File\n\nThe Log file contains all data needed for the system identification of the specified model as defined in its config file. Next to the [ULog](https://docs.px4.io/master/en/dev_log/ulog_file_format.html) file format it is also possible to provide the data as a csv file. An example of the required formating can be seen in the `resources` folder.\n\n#### Data Selection\n\nThe data_selection argument is optional (per default none) and can be used to visually select subportions of the data.\n\n- none(default): Data selection is disabled, and the whole section of the log is used\n- interactive: Data is selected interactively using the [Visual Dataframe Selector](https://github.com/manumerous/visual_dataframe_selector), before running the model estimation. It is also possible to save the selected subportion of data to a csv file in order to use this exact dataset multiple times.\n- setpoint: Data is selected based on a certain topic value, which has to be specified with the variable `selection_var` through the command line. The variable has to be provided in the format `topic_name/variable_name` to be recognized and loaded correctly. For example, to select data based on the `aux1` value of the `manual_control_setpoint` topic, the variable `manual_control_setpoint/aux1` has to be specified.\n- auto: Data is selected automatically (Beta)\n\n### Results\n\nThe resulting parameters of the model estimation together with additional report information will be saved into the `model_results` folder as a yaml file.\n\n### Getting Started\n\nAs an example to get started you estimate the parameters of a quadrotor model with the reference log_files:\n\n```\nmake estimate-model model=quadrotor_model log=resources/quadrotor_model.ulg\n```\n\n## Generating a Model Prediction for Given Parameters and Log\n\nIt is also possible to test the obtained parameters for a certain model on a different log using:\n\n```\nmake predict-model [model=\u003cmodeltype\u003e] [config=\u003cconfig_file_path\u003e] [data_selection=\u003cnone|interactive|auto\u003e] [log=\u003clog_file_path\u003e] [model_results=\u003cmodel_results_path\u003e]\n```\n\n## Testing the functionality of Parametric model\n\nTo ensure that the parametric model works as expected you can perform a set of pytests, which are stored in `Tools/parametric_model/tests`. To start the tests you have to run the shell script:\n\n`Tools/parametric_model/test_parametric_model.sh`\n\nCurrently only the transformation from body to intertial frame and vise versa are checked. This should be expanded in the future.\n\n## Running the Simulation\n\nTo run the simulation,\n\n```\nsource setup.bash\nTools/sitl_run.sh -m iris -s iris_aerodynamics\n```\n\nThe custom Gazebo quadrotor model will always read the model parameters from the file `model_results/quadrotor_model.yaml`. You can simply rename your desired model results file to fly your estimated model in Gazebo.\n\n## Credits\n\nThis project was done in collaboration between the [Autonomous Systems Lab, ETH Zurich](https://asl.ethz.ch/) and [Auterion AG](https://auterion.com/)\n\nTo cite this work in a academic context:\n\n```\n@article{galliker2021data,\n  title={Data-Driven Dynamics Modelling Using Flight Logs},\n  author={Galliker, Manuel Yves},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethz-asl%2Fdata-driven-dynamics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethz-asl%2Fdata-driven-dynamics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethz-asl%2Fdata-driven-dynamics/lists"}