{"id":16358994,"url":"https://github.com/wil3/gymfc","last_synced_at":"2025-04-05T15:06:12.035Z","repository":{"id":47369115,"uuid":"128968861","full_name":"wil3/gymfc","owner":"wil3","description":"A universal flight control tuning framework","archived":false,"fork":false,"pushed_at":"2021-10-07T16:59:11.000Z","size":8518,"stargazers_count":415,"open_issues_count":9,"forks_count":103,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-03-29T14:08:02.480Z","etag":null,"topics":["benchmark","drone","flight-controller","gazebo","gazebo-plugin","gazebo-simulator","machinelearning","openai","openai-gym","openai-gym-environments","quadcopter","reinforcement-learning","rl","robotics","uav"],"latest_commit_sha":null,"homepage":"http://wfk.io/neuroflight/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wil3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-10T17:11:12.000Z","updated_at":"2025-03-20T10:16:35.000Z","dependencies_parsed_at":"2022-08-21T19:20:12.150Z","dependency_job_id":null,"html_url":"https://github.com/wil3/gymfc","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wil3%2Fgymfc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wil3%2Fgymfc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wil3%2Fgymfc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wil3%2Fgymfc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wil3","download_url":"https://codeload.github.com/wil3/gymfc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247353732,"owners_count":20925329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","drone","flight-controller","gazebo","gazebo-plugin","gazebo-simulator","machinelearning","openai","openai-gym","openai-gym-environments","quadcopter","reinforcement-learning","rl","robotics","uav"],"created_at":"2024-10-11T02:07:14.721Z","updated_at":"2025-04-05T15:06:12.003Z","avatar_url":"https://github.com/wil3.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![GymFC](https://github.com/wil3/gymfc/blob/master/images/gymfc-logo.png)\n[![Build Status](https://travis-ci.com/wil3/gymfc.svg?branch=master)](https://travis-ci.com/wil3/gymfc)\n[![All-Contributors](https://img.shields.io/badge/dynamic/json?color=orange\u0026label=all%20contributors\u0026query=%24.contributors.length\u0026url=https%3A%2F%2Fraw.githubusercontent.com%2Fwil3%2Fgymfc%2Fmaster%2F.all-contributorsrc)](#contributors-)\n\nGymFC is flight control tuning framework with a focus in attitude control. \nGymFC was first introduced in the [manuscript](http://wfk.io/docs/gymfc.pdf) \"Reinforcement learning for UAV attitude control\" in which a simulator was used to\nsynthesize neuro-flight attitude controllers that exceeded the performance of a traditional PID controller. \nSince the projects initial release it has matured to become a modular\nframework\nfor tuning flight control systems, not only for synthesizing neuro-flight\ncontrollers but also tuning traditional controllers as well. \nGymFC is the primary method for developing controllers to be used in the worlds\nfirst neural network supported\nflight control firmware [Neuroflight](https://wfk.io/neuroflight). \nDetails of the project and its architecture are best described in Wil Koch's \n[thesis](http://wfk.io/docs/WilliamKochThesisFINAL.pdf) \"Flight Controller Synthesis Via Deep Reinforcement Learning\".\n\nPlease use the following BibTex entries to cite our work,\n```\n@article{koch2019reinforcement,\n  title={Reinforcement learning for UAV attitude control},\n  author={Koch, William and Mancuso, Renato and West, Richard and Bestavros, Azer},\n  journal={ACM Transactions on Cyber-Physical Systems},\n  volume={3},\n  number={2},\n  pages={22},\n  year={2019},\n  publisher={ACM}\n}\n```\n```\n@article{koch2019flight,\n  title={Flight Controller Synthesis Via Deep Reinforcement Learning},\n  author={Koch, William},\n  journal={arXiv preprint arXiv:1909.06493},\n  year={2019}\n}\n```\n\n![Architecture](https://github.com/wil3/gymfc/blob/master/images/gymfc2-arch.png)\n\n\n## Table of contents\n\n* [Features](https://github.com/wil3/gymfc#features)\n* [News](https://github.com/wil3/gymfc#news)\n* [Installation](https://github.com/wil3/gymfc#installation)\n* [Getting Started](https://github.com/wil3/gymfc#getting-started)\n* [Tutorials](https://github.com/wil3/gymfc#tutorials)\n* [User Modules](https://github.com/wil3/gymfc#available-user-provided-modules)\n* [Custom Modules](https://github.com/wil3/gymfc#custom-user-modules)\n* [Examples](https://github.com/wil3/gymfc#examples)\n* [Development Team](https://github.com/wil3/gymfc#)\n* [Contributions](https://github.com/wil3/gymfc#contributions)\n\n\n\n\n# Features\n\n* Support for IMU, ESC and battery sensors\n* Aircraft agnostic - support for any type of aircraft just configure number of\n  actuators and sensors.\n* Digital twin independence - digital twin is developed external to GymFC\n  allowing separate versioning.\n* Google protobuf aircraft digital twin API for publishing control\n  signals and subscribing to sensor data. \n* Flexible agent interface allowing controller development for any type of flight control systems.\n* Support for Gazebo 8, 9, and 11. Gazebo plugins are built dynamically depending on\n  your installed version. \n\n\n# News\n* May 2020 - [NF1](https://rotorbuilds.com/build/15163) quadcopter model and reward functions used in this thesis work is published in the `examples` directory.\n* August 2019 - GymFC synthesizes neuro-controller with [new level of\n  performance](https://www.youtube.com/watch?v=MByCyEnsYP0).  \n* August 2019 - Thesis is defended, [Flight Controller Synthesis via Deep\n  Reinforcement Learning](http://wfk.io/docs/WilliamKochThesisFINAL.pdf). \n* July 2019 - GymFC v0.2.0 is released. \n* December 2018 - Our GymFC manuscript is accepted to the journal ACM Transactions on Cyber-Physical Systems.\n* November 2018 - Flight controller synthesized with GymFC achieves stable\n  flight in [Neuroflight](https://github.com/wil3/neuroflight).\n* September 2018 - GymFC v0.1.0 is released.\n* April 2018 - Pre-print of our paper is published to [arXiv](https://arxiv.org/abs/1804.04154). \n\n\n# Installation \n\n## Quick start\nTo install GymFC and its dependencies on Ubuntu 18.04 execute,\n```\nsudo MAKE_FLAGS=-j4 ./install_dependencies.sh\npip3 install .\n```\n\n## Dependencies\nGymFC runs on Ubuntu 18.04 and uses  [Gazebo v10.1.0](http://gazebosim.org/tutorials?tut=install_from_source\u0026cat=install) with [Dart v6.7.0](https://github.com/dartsim/dart/tree/v6.7.0) for the backend simulator. To use Dart with Gazebo, they must be installed from source. For why Gazebo must be used with Dart see this [video](https://www.youtube.com/watch?v=d3NyFU0bVT0). The easiest way to install the dependencies is with the provided `install_dependencies.sh` script. By default it will run `make` with a single job. You can override the `make` flags with the `MAKE_FLAGS` environment variable. Building Gazebo from source is very resource intensive. If you have sufficient memory increase the number of jobs to run in parallel. For example to run four jobs in parallel execute, \n```\nsudo MAKE_FLAGS=-j4 ./install_dependencies.sh\n```\nNote, this script may take more than an hour to execute. If your build fails\ncheck `dmesg` but the most common reason will be out-of-memory failures. \n\n## GymFC\n(Optional) It is suggested to set up a [virtual environment](https://docs.python.org/3/library/venv.html) to install GymFC into. From the project root run,\n   `python3 -m venv env`. This will create an environment named `env` which\nwill be ignored by git. To enable the virtual environment, `source\nenv/bin/activate` and to deactivate, `deactivate`.  \n\nInstall GymFC,\n```\npip3 install .\n```\nThis will install the Python dependencies and also build the Gazebo plugins and\nmessages. \n\n## Developing with GymFC\nIf you plan to modify the GymFC code you will need to install in\nedit/development mode. \n```\npip3 install -e .\n```\nYou will also have to manually install the Gazebo plugins by executing,\n```\ngymfc/envs/assets/gazebo/plugins/build_plugin.sh\n```\nIf you deviate from this installation instructions (e.g., installing Gazebo in\na different location other than specific in `install_dependencies.sh`), you\nmay need to change the location of the Gazebo `setup.sh` defined by the\nvariable SetupFile in `gymfc/gymfc.ini`.\n\n## Verifying install\n\nGymFC requires an aircraft model (digital twin) to run. The NF1 racing\nquadcopter model is available in `examples/gymfc_nf/twins/nf1` if you need a\nmodel for testing. To test everything is installed correctly run,\n\n```\npython3 tests/test_start_sim.py --verbose examples/gymfc_nf/twins/nf1/model.sdf\n```\nIf everything is OK you should see the NF1 quadcopter model in Gazebo.\n\nYou will see the following error message because you have not built the\nmotor and IMU plugins yet.\n```\n[Err] [Plugin.hh:187] Failed to load plugin libgazebo_motor_model.so: libgazebo_motor_model.so: cannot open shared object file: No such file or directory\n[Err] [Plugin.hh:187] Failed to load plugin libgazebo_imu_plugin.so: libgazebo_imu_plugin.so: cannot open shared object file: No such file or directory\n```\nAlso the following error message is normal,\n``` \n[Err] [DARTJoint.cc:195] DARTJoint: SetAnchor is not implemented\n```\n\nTo use the NF1 model for further testing read examples/README.md. \n\n## Install by Docker\nThis repository includes an experimental docker build in `docker/demo` that demos the usage of GymFC. \nIt has been tested on MacOS 10.14.3 and Ubuntu 18.04, however the Gazebo client\nhas not been verified to work for Ubuntu. This docker image can help ensure you\nare running a supported environment for GymFC.\n\n### Install dependencies \n\nFor Mac, install [Docker for Mac](https://docs.docker.com/docker-for-mac/install/) and [XQuartz](https://www.xquartz.org/) on your system.\nFor Ubuntu, install [Docker for Ubuntu](https://docs.docker.com/engine/install/ubuntu/).\n\n### Build and test \nBuild the docker image\n\n```bash\ndocker build  . -t gymfc:demo\n```\nThis will take a while as it compiles mesa drivers, gazebo and dart. It is recommended to give Docker a large part of the host's resources.\nAll incoming connections will forward to xquartz:\n\n```bash\nxhost +\n```\n\nExample usage, run the image and test test_step_sim.py using the [Solo digital twin](https://github.com/wil3/gymfc-digitaltwin-solo.git),\n```bash\ndocker run -ti -e DISPLAY=\u003chostip\u003e:0 \\\n-v \u003cpath-to-gymfc-digitaltwin-solo\u003e/models/solo/model.sdf:/gymfc/demo/models/solo/model.sdf \\\ngymfc:demo \\\n\"python3 /gymfc/tests/test_step_sim.py --gymfc-config /gymfc/gymfc.ini --verbose /gymfc/demo/models/solo/model.sdf  1 1 1 1\"\n```\n\nReplace _\u003chostip\u003e_ by the external ip of your system to allow gymfc to connect to your XQuartz server and _\u003cpath-to-gymfc-digitaltwin-solo\u003e_ to where you cloned the Solo repo.\nTake special note that the `test_step_sim.py` parameters are using the containers\npath, not the host's path.\n\n\n# Getting Started \n\nThe simplest environment can be created with,\n\n```python\nfrom gymfc.envs.fc_env import FlightControlEnv\nclass MyEnv(FlightControlEnv):\n    def __init__(self, aircraft_config, config=None, verbose=False):\n        super().__init__(aircraft_config, config_filepath=config, verbose=verbose)\n```\n\nBy inheriting FlightControlEnv you now have access to the `step_sim` and\n`reset` functions. If you want to create an OpenAI gym you also need to inherit\nthis class e.g.,\n\n```python\n\nfrom gymfc.envs.fc_env import FlightControlEnv\nimport gym\nclass MyOpenAIEnv(FlightControlEnv, gym.Env):  \n```\n \nFor simplicity the GymFC environment takes as input a single `aircraft_config` which is the file location of your aircraft model  `model.sdf`. The SDF declares all the visualizations, geometries and plugins for the aircraft.\n\n### Directory Layout\nGymFC expects your model to have the following Gazebo style directory structure: \n```\nmodel_name/\n  model.config\n  model.sdf\n  plugins/\n    build/\n```\nwhere the `plugin` directory contains the  source for your plugins and the\n`build` directory will contain the built binary plugins. GymFC will, at\nruntime, add the build directory to the Gazebo plugin path so they can be found and loaded.\n\nNOTE! If you are using external plugins create soft links\nto each .so file in the build directory.\n\n\n# Tutorials\n\n* [Installation](https://www.youtube.com/watch?v=r6Q0Q7z2K9c)\n\nMore coming soon!\n\n\n# Available User Provided Modules\n\nTo increase flexibility and provide a universal tuning framework, the user must\nprovide four modules: A flight controller, a flight control tuner, environment\ninterface, and digital twin. (Note: for neuro-flight controllers typically the\nflight controller and tuner are one in the same, e.g., OpenAI baselines) This will expand the flight control research that\ncan be done with GymFC. For example this opens up the possibilities for tuning\nPID gains using optimization strategies such as GAs and PSO. The goal is to provide a collection of open source\nmodules for users to mix and match. If you have created your own, please let us\nknow and we will add it below.\n \n## Tuners\n\n* [OpenAI baselines](https://github.com/openai/baselines)\n\n## Environments\n\n* [gymfc_nf-step-v1](https://github.com/wil3/gymfc/tree/master/examples/gymfc_nf/envs)\n\n## Digital Twins\n\n* [NF1](https://github.com/wil3/gymfc/tree/master/examples/gymfc_nf/twins/nf1)\n* [Solo](https://github.com/wil3/gymfc-digitaltwin-solo) Needs help!\n\n## Motor models\n\n* [Element blade theory](https://github.com/wil3/gymfc-aircraft-plugins)\n\n\n# Custom User Modules\n\n### Digital Twin \n\n\n## SDF\n\nEach model.sdf **must** declare the `libAircraftConfigPlugin.so` plugin. \nThis is a dummy plugin allowing us to set arbitrary configuration data.\nAn example configuration may look like this,\n\n```xml\n\u003cplugin name=\"config\" filename=\"libAircraftConfigPlugin.so\"\u003e\n    \u003c!-- Define the total number of motors that shall be controlled --\u003e\n    \u003cmotorCount\u003e4\u003c/motorCount\u003e\n\n    \u003c!-- The center of thrust must be defined in order to attach the aircraft\nmodel to the simulation. The offset will in relation to this specified link --\u003e\n    \u003ccenterOfThrust\u003e \n        \u003clink\u003ebattery\u003c/link\u003e\n        \u003coffset\u003e0 0 0.058\u003c/offset\u003e\n    \u003c/centerOfThrust\u003e\n    \u003c!-- Specify all the sensors this aircraft supports. Valid sensor types \nare \"imu, esc, and battery\" --\u003e\n    \u003csensors\u003e\n      \u003csensor type=\"imu\"\u003e\n          \u003cenable_angular_velocity\u003etrue\u003c/enable_angular_velocity\u003e\n          \u003cenable_linear_acceleration\u003etrue\u003c/enable_linear_acceleration\u003e\n          \u003cenable_orientation\u003etrue\u003c/enable_orientation\u003e\n      \u003c/sensor\u003e\n      \u003c!--\n      \u003csensor type=\"esc\"\u003e\n            \u003cenable_angular_velocity\u003etrue\u003c/enable_angular_velocity\u003e\n            \u003cenable_temperature\u003etrue\u003c/enable_temperature\u003e\n            \u003cenable_current\u003etrue\u003c/enable_current\u003e\n      \u003c/sensor\u003e\n      \u003csensor type=\"battery\"\u003e\n          \u003cenable_voltage\u003etrue\u003c/enable_voltage\u003e\n          \u003cenable_current\u003etrue\u003c/enable_current\u003e\n      \u003c/sensor\u003e\n        --\u003e\n    \u003c/sensors\u003e\n\u003c/plugin\u003e\n```\n\n### API\nGymFC communicates with the aircraft through Google Protobuf messages. At a\nminimum the aircraft must subscribe to motor commands and publish IMU messages\n\n#### GymFC to Aircraft\n\n*Topic* /aircraft/command/motor \n*Message Type* [MotorCommand.proto]()\n\n#### Aircraft to GymFC\n\n*Topic* /aircraft/sensor/imu \n\n*Message Type* Imu.proto\n\n\n*Topic* /aircraft/sensor/esc \n\n*Message Type* EscSensor.proto\n\n# Examples\n\nThe OpenAI environment and digital twin models used in Wil Koch's thesis can be found in the\n`examples/` directory. \n\n# Creator\n[Wil Koch](https://wfk.io)\n\n# Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://wfk.io\"\u003e\u003cimg src=\"https://avatars3.githubusercontent.com/u/4587660?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eWil Koch\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/commits?author=wil3\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/wil3/gymfc/commits?author=wil3\" title=\"Documentation\"\u003e📖\u003c/a\u003e \u003ca href=\"#example-wil3\" title=\"Examples\"\u003e💡\u003c/a\u003e \u003ca href=\"#infra-wil3\" title=\"Infrastructure (Hosting, Build-Tools, etc)\"\u003e🚇\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/MichaelManz\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/1280036?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eMichael Friedrich\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/commits?author=MichaelManz\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"#example-MichaelManz\" title=\"Examples\"\u003e💡\u003c/a\u003e \u003ca href=\"#infra-MichaelManz\" title=\"Infrastructure (Hosting, Build-Tools, etc)\"\u003e🚇\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/sidmysore\"\u003e\u003cimg src=\"https://avatars2.githubusercontent.com/u/12174271?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSid Mysore\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/issues?q=author%3Asidmysore\" title=\"Bug reports\"\u003e🐛\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/varunag18\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/1232584?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003evarunag18\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/commits?author=varunag18\" title=\"Tests\"\u003e⚠️\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/xabierolaz\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/31626226?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003exabierolaz\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/commits?author=xabierolaz\" title=\"Tests\"\u003e⚠️\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/SwapnilPande\"\u003e\u003cimg src=\"https://avatars3.githubusercontent.com/u/9200948?v=4\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSwapnil Pande\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/wil3/gymfc/issues?q=author%3ASwapnilPande\" title=\"Bug reports\"\u003e🐛\u003c/a\u003e \u003ca href=\"#ideas-SwapnilPande\" title=\"Ideas, Planning, \u0026 Feedback\"\u003e🤔\u003c/a\u003e \u003ca href=\"https://github.com/wil3/gymfc/commits?author=SwapnilPande\" title=\"Tests\"\u003e⚠️\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-enable --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nWant to become a contributor?! Visit [CONTRIBUTING.md](https://github.com/wil3/gymfc/blob/master/CONTRIBUTING.md) for more information to get started. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwil3%2Fgymfc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwil3%2Fgymfc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwil3%2Fgymfc/lists"}