{"id":18385827,"url":"https://github.com/uzh-rpg/sim2real_drone_racing","last_synced_at":"2025-04-07T00:32:28.775Z","repository":{"id":45556920,"uuid":"216063828","full_name":"uzh-rpg/sim2real_drone_racing","owner":"uzh-rpg","description":"A Framework for Zero-Shot Sim2Real Drone Racing ","archived":false,"fork":false,"pushed_at":"2020-07-17T12:04:07.000Z","size":29022,"stargazers_count":97,"open_issues_count":11,"forks_count":24,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-22T09:43:23.398Z","etag":null,"topics":["deep-learning","neural-network","robotics","sim2real"],"latest_commit_sha":null,"homepage":"http://rpg.ifi.uzh.ch/research_drone_racing.html","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uzh-rpg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-18T16:19:56.000Z","updated_at":"2025-03-13T17:45:39.000Z","dependencies_parsed_at":"2022-08-31T04:10:53.838Z","dependency_job_id":null,"html_url":"https://github.com/uzh-rpg/sim2real_drone_racing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uzh-rpg%2Fsim2real_drone_racing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uzh-rpg%2Fsim2real_drone_racing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uzh-rpg%2Fsim2real_drone_racing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uzh-rpg%2Fsim2real_drone_racing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uzh-rpg","download_url":"https://codeload.github.com/uzh-rpg/sim2real_drone_racing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247574088,"owners_count":20960495,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","neural-network","robotics","sim2real"],"created_at":"2024-11-06T01:19:14.909Z","updated_at":"2025-04-07T00:32:25.558Z","avatar_url":"https://github.com/uzh-rpg.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Drone Racing: From Simulation to Reality with Domain Randomization\n\nThis repo contains the implementation of a zero-shot sim2real method for drone racing.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./docs/sim2real.gif\" alt=\"ddr\"\u003e\n\u003c/p\u003e\n\nFor more information visit the project page:[http://rpg.ifi.uzh.ch/research\\_drone\\_racing.html](http://rpg.ifi.uzh.ch/research_drone_racing.html).\n\n#### Citing\n\nIf you use this code in an academic context, please cite the following publication:\n\nPaper: [Deep Drone Racing: From Simulation to Reality with Domain Randomization](http://rpg.ifi.uzh.ch/docs/TRO19_Loquercio.pdf)\n\nVideo: [YouTube](https://youtu.be/vdxB89lgZhQ)\n\n```\n@article{loquercio2019deep,\n  title={Deep Drone Racing: From Simulation to Reality with Domain Randomization},\n  doi={10.1109/TRO.2019.2942989},\n  author={Loquercio, Antonio and Kaufmann, Elia and Ranftl, Ren{\\'e} and Dosovitskiy, Alexey and Koltun, Vladlen and Scaramuzza, Davide},\n  journal={IEEE Transactions on Robotics},\n  year={2019}\n}\n\n```\n## Installation\n\n### Requirements\n\nThe code was tested with Ubuntu 18.04 and ROS Melodic.\nDifferent OS and ROS versions are possible but not supported.\n\n### Step-by-Step Procedure\n\nUse the following commands to create a new catkin workspace and a virtual environment with all the required dependencies.\n\n```bash\nexport ROS_VERSION=melodic\nmkdir drone_racing_ws\ncd drone_racing_ws\nexport CATKIN_WS=./catkin_ddr\nmkdir -p $CATKIN_WS/src\ncd $CATKIN_WS\ncatkin init\ncatkin config --extend /opt/ros/$ROS_VERSION\ncatkin config --merge-devel\ncatkin config --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color\ncd src\n\ngit clone https://github.com/uzh-rpg/sim2real_drone_racing.git\ncd sim2real_drone_racing\ncd ..\nvcs-import \u003c sim2real_drone_racing/dependencies.yaml\ntouch octomap/octovis/CATKIN_IGNORE\n\n# Build and re-source the workspace\ncatkin build\n. $CATKIN_WS/devel/setup.bash\n\n# Create your learning environment\nvirtualenv -p python2.7 ./droneflow\nsource ./droneflow/bin/activate\n\n# If you have a GPU, use the following command. You will need to have CUDA 10.0 installed for it to work.\n#pip install tensorflow-gpu==1.13.1\n# If you don't have a GPU, uncomment the previous line and comment the next\npip install tensorflow==1.13.1\n\n# Install Required Python dependecies\ncd $CATKIN_WS/src/sim2real_drone_racing\npip install -r python_dependencies.txt\n\n```\n\n\n## Let's Race\n\nOnce you have installed the dependencies, you will be able to fly in simulation with our pre-trained checkpoint. You don't need GPU for execution. Note that if the network can't run at least at 10Hz, you won't be able to fly successfully.\n\nOpen a terminal and type:\n```bash\ncd drone_racing_ws\n. ./catkin_ddr/devel/setup.bash\n. ./droneflow/bin/activate\nexport CUDA_VISIBLE_DEVICES=''\nroslaunch deep_drone_racing_learning  net_controller_launch.launch\n\n```\n\nOpen an other terminal and type:\n```bash\ncd drone_racing_ws\n. ./catkin_ddr/devel/setup.bash\n. ./droneflow/bin/activate\nroslaunch test_racing test_racing.launch\n\n```\n\n## Train your own Sim2Real model\n\nYou can use the following commands to generate data in simulation and train your model on it. The trained checkpoint can then be used to control a physical platform on a race track.\n\n### Generate data\n\n```bash\ncd drone_racing_ws\n. ./catkin_ddr/devel/setup.bash\n. ./droneflow/bin/activate\nroscd drone_racing/resources/scripts\npython collect_data.py\n\n```\n\nIt is possible to change parameters (number of iteration per background/ gate texture/ etc. ) in the above script.\nDefaults should be good. Optionally, you can use the data we have already collected, available at [this link](http://rpg.ifi.uzh.ch/datasets/sim2real_ddr/simulation_training_data.zip).\n\n\n### Train the Network\n\n```bash\nroscd deep_drone_racing_learner/src/ddr_learner\n\n```\n\nModify the file [train\\_model.sh](./learning/deep_drone_racing_learner/src/ddr_learner/train_model.sh) to add the path of validation data collected in the real world, which you can download from [this link](http://rpg.ifi.uzh.ch/datasets/sim2real_ddr/validation_real_data.zip).\nThen, run the following command to train the model.\n\n```bash\n./train_model.sh\n\n```\n\n### Test the Network\n\nEdit the following file to use the checkpoint you just trained\n\n```bash\nrosed deep_drone_racing_learning net_controller_launch.launch\n\n```\n\nThe trained network can now be tested in an environment which was never observed at training time.\n\nOpen a terminal and run:\n\n```bash\ncd drone_racing_ws\n. ./catkin_ddr/devel/setup.sh\n. ./droneflow/bin/activate\nexport CUDA_VISIBLE_DEVICES=''\nroslaunch deep_drone_racing_learning  net_controller_launch.launch\n\n```\n\nOpen another terminal and run:\n\n```bash\ncd drone_racing_ws\n. ./catkin_ddr/devel/setup.sh\n. ./droneflow/bin/activate\nroslaunch test_racing test_racing.launch\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuzh-rpg%2Fsim2real_drone_racing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuzh-rpg%2Fsim2real_drone_racing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuzh-rpg%2Fsim2real_drone_racing/lists"}