{"id":31079505,"url":"https://github.com/jacopopan/aerial-autonomy-stack","last_synced_at":"2025-09-16T10:22:52.243Z","repository":{"id":312425640,"uuid":"1005678829","full_name":"JacopoPan/aerial-autonomy-stack","owner":"JacopoPan","description":"ROS2 multi-drone PX4 and ArduPilot SITL with YOLO—using Dockerized simulation and deployment for Jetson","archived":false,"fork":false,"pushed_at":"2025-09-07T14:40:23.000Z","size":26429,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-07T16:29:08.548Z","etag":null,"topics":["ardupilot","docker","drone","fixed-wing","gz-harmonic","jetpack6","jetson","mavros","multicopter","onnxruntime-gpu","px4","quadrotor","ros2","sim2real","simulation","sitl","vtol","xrce-dds","yolov8","zenoh"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JacopoPan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-20T16:11:46.000Z","updated_at":"2025-09-07T14:40:26.000Z","dependencies_parsed_at":"2025-08-30T15:22:57.207Z","dependency_job_id":"68bd49a3-418c-4838-82cd-aefb34fe5b5d","html_url":"https://github.com/JacopoPan/aerial-autonomy-stack","commit_stats":null,"previous_names":["jacopopan/aerial-autonomy-stack"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/JacopoPan/aerial-autonomy-stack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacopoPan%2Faerial-autonomy-stack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacopoPan%2Faerial-autonomy-stack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacopoPan%2Faerial-autonomy-stack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacopoPan%2Faerial-autonomy-stack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JacopoPan","download_url":"https://codeload.github.com/JacopoPan/aerial-autonomy-stack/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacopoPan%2Faerial-autonomy-stack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275401149,"owners_count":25458091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ardupilot","docker","drone","fixed-wing","gz-harmonic","jetpack6","jetson","mavros","multicopter","onnxruntime-gpu","px4","quadrotor","ros2","sim2real","simulation","sitl","vtol","xrce-dds","yolov8","zenoh"],"created_at":"2025-09-16T10:22:48.707Z","updated_at":"2025-09-16T10:22:52.234Z","avatar_url":"https://github.com/JacopoPan.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# aerial-autonomy-stack\n\n*Aerial autonomy stack* (AAS) is a software stack to:\n\n1. **Develop** end-to-end drone autonomy with ROS2\n2. **Simulate** vision and control in software-in-the-loop, with YOLOv8 and PX4/ArduPilot\n3. **Deploy** in real drones with NVIDIA Orin/JetPack\n\n\u003e For the motivation behind AAS and how it compares to similar projects, read [`RATIONALE.md`](/supplementary/RATIONALE.md)\n\nhttps://github.com/user-attachments/assets/c194ada6-2996-4bfa-99e9-32b45e29281d\n\n## Features\n\n- Support for multiple **quadrotors and VTOLs** based on either **PX4 or ArduPilot**\n- **ROS2**-based autopilot interfaces (*via* XRCE-DDS and MAVROS)\n- Support for **YOLOv8** (with ONNX GPU Runtimes) and **LiDAR Odometry** (with [KISS-ICP](https://github.com/PRBonn/kiss-icp))\n- **Dockerized simulation** based on [`nvcr.io/nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04`](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags)\n- **Dockerized deployment** based on [`nvcr.io/nvidia/l4t-jetpack:r36.4.0`](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-jetpack/tags)\n- **3D worlds** for [PX4](https://docs.px4.io/main/en/simulation/#sitl-simulation-environment) and [ArduPilot](https://ardupilot.org/dev/docs/sitl-simulator-software-in-the-loop.html#sitl-architecture) software-in-the-loop (SITL) simulation\n- [Zenoh](https://github.com/eclipse-zenoh/zenoh-plugin-ros2dds) inter-vehicle ROS2 bridge\n- Support for [PX4 Offboard](https://docs.px4.io/main/en/flight_modes/offboard.html) mode (e.g. CTBR/`VehicleRatesSetpoint` for agile, GNSS-denied flight) \n- Support for [ArduPilot Guided](https://ardupilot.org/copter/docs/ac2_guidedmode.html) mode (i.e. `setpoint_velocity`, `setpoint_accel` references)\n- Logs analysis with [`flight_review`](https://github.com/PX4/flight_review) (`.ulg`), MAVExplorer (`.bin`), and [PlotJuggler](https://github.com/facontidavide/PlotJuggler) (`rosbag`)\n- **Steppable simulation** interface for reinforcement learning\n- Support for Gazebo's [**`WindEffects`**](https://github.com/gazebosim/gz-sim/blob/gz-sim10/examples/worlds/wind.sdf) (except PX4 VTOL)\n\n\u003cdetails\u003e\n\u003csummary\u003eAAS leverages the following frameworks: \u003ci\u003e(expand)\u003c/i\u003e\u003c/summary\u003e\n\n\u003e [*ROS2 Humble*](https://docs.ros.org/en/rolling/Releases.html) (LTS, EOL 5/2027), [*Gazebo Sim Harmonic*](https://gazebosim.org/docs/latest/releases/) (LTS, EOL 9/2028), [*PX4 1.16*](https://github.com/PX4/PX4-Autopilot/releases) interfaced *via* [XRCE-DDS](https://github.com/eProsima/Micro-XRCE-DDS/releases), [*ArduPilot 4.6*](https://github.com/ArduPilot/ardupilot/releases) interfaced *via* [MAVROS](https://github.com/mavlink/mavros/releases), [*YOLOv8*](https://github.com/ultralytics/ultralytics/releases) on [*ONNX Runtime 1.22*](https://onnxruntime.ai/getting-started) (latest stable releases as of 8/2025), [*L4T 36* (Ubuntu 22-based)/*JetPack 6*](https://developer.nvidia.com/embedded/jetpack-archive) (for deployment only, latest major release as of 8/2025)\n\n\u003c/details\u003e\n\n---\n\n## Part 1: Installation of AAS\n\n\u003e [!IMPORTANT]\n\u003e This stack is developed and tested using a [Ubuntu 22.04](https://ubuntu.com/about/release-cycle) host (penultimate LTS, ESM 4/2032) with [**`nvidia-driver-575`**](https://developer.nvidia.com/datacenter-driver-archive) and Docker Engine v28 (latest stable releases as of 7/2025) on an i9-13 with RTX3500 and an i7-11 with RTX3060—**note that an NVIDIA GPU *is* required**\n\u003e \n\u003e **To setup the requirements: (i) Ubuntu 22, Git LFS, (ii) NVIDIA driver, (iii) Docker Engine, (iv) NVIDIA Container Toolkit, and (v) NVIDIA NGC API Key, read [`PREINSTALL.md`](/supplementary/PREINSTALL.md)**\n\n```sh\n# Clone this repo\nmkdir -p ~/git\ngit clone git@github.com:JacopoPan/aerial-autonomy-stack.git ~/git/aerial-autonomy-stack\ncd ~/git/aerial-autonomy-stack\n```\n\n### Build the Docker Images\n\n\u003e [!WARNING]\n\u003e The build script creates two ~20GB images (including lots of tools and artifacts for development)\n\u003e \n\u003e Building from scratch requires a good/stable internet connection (`Ctrl + c` and restart if necessary)\n\n```sh\n# Clone external repos (in github_clones/) and build the Docker images\ncd ~/git/aerial-autonomy-stack/scripts\n./sim_build.sh # The first build takes ~25', subsequent ones take seconds to minutes\n```\n\n---\n\n## Part 2: Simulation and Development with AAS\n\n```sh\n# Start a simulation (note: ArduPilot STIL takes ~40s to be ready to arm)\ncd ~/git/aerial-autonomy-stack/scripts\nAUTOPILOT=px4 NUM_QUADS=1 NUM_VTOLS=1 WORLD=swiss_town ./sim_run.sh # Check the script for more options\n```\n\n\u003e On a low-mid range laptop—i7-11 with 16GB RAM and RTX3060—AAS simulates 3 PX4 quads with camera and LiDAR at 99% of the wall-clock (note that ArduPilot faster physics updates and more complex worlds have higher computational demands)\n\u003e\n\u003e Once \"Ready to Fly\", one can takeoff and control from QGroundControl's [\"Fly View\"](https://docs.qgroundcontrol.com/master/en/qgc-user-guide/fly_view/fly_view.html)\n\n![worlds](https://github.com/user-attachments/assets/45a2f2ad-cc31-4d71-aa2e-4fe542a59a77)\n\nAvailable `WORLD`s:\n- `apple_orchard`, a GIS world created using [BlenderGIS](https://github.com/domlysz/BlenderGIS)\n- `impalpable_greyness`, (default) an empty world with simple shapes\n- `shibuya_crossing`, a 3D world adapted from [cgtrader](https://www.cgtrader.com/)\n- `swiss_town`, a photogrammetry world courtesy of [Pix4D / pix4d.com](https://support.pix4d.com/hc/en-us/articles/360000235126)\n\nTo advance the simulation in **discrete time steps**, e.g. 1s, from a terminal on the host, run:\n\n```sh\ndocker exec simulation-container bash -c \"gz service -s /world/\\$WORLD/control --reqtype gz.msgs.WorldControl --reptype gz.msgs.Boolean --req 'multi_step: 250, pause: true'\" # Adjust multi_step based on the value of max_step_size in the world's .sdf (defaults: 250 for PX4, 1000 for ArduPilot)\n```\n\nTo add or disable a **wind field**, from a terminal on the host, run:\n\n```sh\ndocker exec simulation-container bash -c \"gz topic -t /world/\\$WORLD/wind/ -m gz.msgs.Wind  -p 'linear_velocity: {x: 0.0 y: 3.0}, enable_wind: true'\" # Positive X blows from the West, positive Y blows from the South\n\ndocker exec simulation-container bash -c \"gz topic -t /world/\\$WORLD/wind/ -m gz.msgs.Wind  -p 'enable_wind: false'\" # Disable WindEffects\n```\n\n\u003e [!TIP]\n\u003e \u003cdetails\u003e\n\u003e \u003csummary\u003eTmux and Docker Shortcuts \u003ci\u003e(expand)\u003c/i\u003e\u003c/summary\u003e\n\u003e \n\u003e - Move between Tmux windows with `Ctrl + b`, then `n`, `p`\n\u003e - Move between Tmux panes with `Ctrl + b`, then `arrow keys`\n\u003e - Enter copy mode to scroll back with `Ctrl + [`, then `arrow keys`, exit with `q`\n\u003e - Split a Tmux window with `Ctrl + b`, then `\"` (horizontal) or `%` (vertical)\n\u003e - Detach Tmux with `Ctrl + b`, then `d`\n\u003e ```sh\n\u003e tmux list-sessions # List all sessions\n\u003e tmux attach-session -t [session_name] # Reattach a session\n\u003e tmux kill-session -t [session_name] # Kill a session\n\u003e tmux kill-server # Kill all sessions\n\u003e ```\n\u003e Docker hygiene:\n\u003e ```sh\n\u003e docker ps -a # List containers\n\u003e docker stop $(docker ps -q) # Stop all containers\n\u003e docker container prune # Remove all stopped containers\n\u003e \n\u003e docker images # List images\n\u003e docker image prune # Remove untagged images\n\u003e docker rmi \u003cimage_name_or_id\u003e # Remove a specific image\n\u003e docker builder prune # Clear the cache system wide\n\u003e ```\n\u003e \u003c/details\u003e\n\n### Fly a Mission\n\n```sh\ncd ~/git/aerial-autonomy-stack/scripts\nAUTOPILOT=px4 NUM_QUADS=1 ./sim_run.sh # Also try AUTOPILOT=ardupilot, or NUM_QUADS=0 NUM_VTOLS=1\n\n# In aircraft 1's terminal\nros2 run mission mission --conops yalla --ros-args -r __ns:=/Drone$DRONE_ID -p use_sim_time:=true # This mission is a simple takeoff, followed by an orbit, and landing for any vehicle\n\n# Finally, in the simulation's terminal\n/simulation_resources/patches/plot_logs.sh # Analyze the flight logs\n```\n\n### Command Line Interface\n\nRead the banner comment in the `autopilot_interface` headers for command line examples (takeoff, orbit, reposition, offboard, land):\n\n- [`ardupilot_interface.hpp`](/aircraft/aircraft_ws/src/autopilot_interface/src/ardupilot_interface.hpp): ArduPilot actions and services\n- [`px4_interface.hpp`](/aircraft/aircraft_ws/src/autopilot_interface/src/px4_interface.hpp): PX4 actions and services\n\nOnce flown from CLI, implemented your mission in [`MissionNode.conops_callback()`](/aircraft/aircraft_ws/src/mission/mission/mission_node.py)\n\n\n### Development\n\nLaunching the `sim_run.sh` script with `MODE=dev`, does **not** start the simulation and mounts folders `simulation_resources`, `aircraft_resources`, and `ros2_ws/src` as volumes to more easily track, commit, push changes while building and testing them within the containers\n\n```sh\n# Develop within live containers\ncd ~/git/aerial-autonomy-stack/scripts\nMODE=dev ./sim_run.sh # Images are pre-built but the ros2_ws/src/ and *_resources/ folders are mounted from the host\n```\n\n\u003e [!NOTE]\n\u003e Project Structure\n\u003e \n\u003e ```sh\n\u003e aerial-autonomy-stack\n\u003e │\n\u003e ├── aircraft\n\u003e │   ├── aircraft_ws\n\u003e │   │   └── src\n\u003e │   │       ├── autopilot_interface # Ardupilot/PX4 high-level actions (Takeoff, Orbit, Offboard, Land)\n\u003e │   │       ├── mission             # Orchestrator of the actions in `autopilot_interface` \n\u003e │   │       ├── offboard_control    # Low-level references for the Offboard action in `autopilot_interface` \n\u003e │   │       ├── state_sharing       # Publisher of the `/state_sharing_drone_N` topic broadcasted by Zenoh\n\u003e │   │       └── yolo_inference      # GStreamer video acquisition and publisher of YOLO bounding boxes\n\u003e │   │\n\u003e │   └── aircraft.yml.erb            # Aircraft docker tmux entrypoint\n\u003e │\n\u003e ├── scripts\n\u003e │   ├── docker\n\u003e │   │   ├── Dockerfile.aircraft     # Docker image for aircraft simulation and deployment\n\u003e │   │   └── Dockerfile.simulation   # Docker image for Gazebo and SITL simulation\n\u003e │   │\n\u003e │   ├── deploy_build.sh             # Build `Dockerfile.aircraft` for arm64/Orin\n\u003e │   ├── deploy_run.sh               # Start the aircraft docker on arm64/Orin\n\u003e │   │\n\u003e │   ├── sim_build.sh                # Build both dockerfiles for amd64/simulation\n\u003e │   └── sim_run.sh                  # Start the simulation\n\u003e │\n\u003e └── simulation\n\u003e     ├── simulation_resources\n\u003e     │   ├── aircraft_models\n\u003e     │   │   ├── alti_transition_quad # ArduPilot VTOL\n\u003e     │   │   ├── iris_with_ardupilot  # ArduPilot quad\n\u003e     │   │   ├── sensor_camera\n\u003e     │   │   ├── sensor_lidar\n\u003e     │   │   ├── standard_vtol        # PX4 VTOL\n\u003e     │   │   └── x500                 # PX4 quad\n\u003e     │   └── simulation_worlds\n\u003e     │       ├── apple_orchard.sdf\n\u003e     │       ├── impalpable_greyness.sdf\n\u003e     │       ├── shibuya_crossing.sdf\n\u003e     │       └── swiss_town.sdf\n\u003e     │\n\u003e     ├── simulation_ws\n\u003e     │   └── src\n\u003e     │       └── ground_system        # Publisher of topic `/tracks` broadcasted by Zenoh\n\u003e     │\n\u003e     └── simulation.yml.erb           # Simulation docker tmux entrypoint\n\u003e ```\n\n---\n\n## Part 3: Deployment of AAS\n\n\u003e [!IMPORTANT]\n\u003e These instructions are tested on a [Holybro Jetson Baseboard](https://holybro.com/products/pixhawk-jetson-baseboard) kit that includes (i) a Pixhawk 6X autopilot and (ii) an NVIDIA Orin NX 16GB computer connected via both serial and ethernet\n\u003e \n\u003e **To setup (i) PX4's DDS UDP client, (ii) ArduPilot serial MAVLink bridge, (iii) JetPack 6, (iv) Docker Engine, (v) NVIDIA Container Toolkit, and (vi) NVIDIA NGC API Key on Orin, read [`AVIONICS.md`](/supplementary/AVIONICS.md)**\n\u003e\n\u003e The Holybro Jetson Baseboard comes with an (i) integrated 4-way (Orin, 6X, RJ-45, JST) Ethernet switch and (ii) two JST USB 2.0 that can be connected to ASIX Ethernet adapters to create additional network interfaces\n\u003e \n\u003e Make sure to configure Orin, 6X's XRCE-DDS, IP radio, Zenoh, etc. consistently with your network setup; the camera acquisition pipeline should be setup in `yolo_inference_node.py`, the LiDAR should publish on topic `/lidar_points` for KISS-ICP (if necessary, discuss in the [Issues](https://github.com/JacopoPan/aerial-autonomy-stack/issues))\n\n\n```sh\n# On Jetson Orin NX, build for arm64 with TensorRT support\nmkdir -p ~/git\ngit clone git@github.com:JacopoPan/aerial-autonomy-stack.git ~/git/aerial-autonomy-stack\ncd ~/git/aerial-autonomy-stack/scripts\n./deploy_build.sh # The first build takes ~1h (mostly to build onnxruntime-gpu from source)\n```\n\n```sh\n# On Jetson Orin NX, start and attach the aerial-autonomy-stack (e.g., from ssh)\nDRONE_TYPE=quad AUTOPILOT=px4 DRONE_ID=1 CAMERA=true LIDAR=false ./deploy_run.sh\ndocker exec -it aircraft-container tmux attach\n```\n\n---\n\u003e You've done a man's job, sir. I guess you're through, huh?\n\n\u003c!-- \n\n## TODOs\n\n- https://developer.nvidia.com/embedded/learn/tutorials/first-picture-csi-usb-camera\n- https://github.com/Livox-SDK/livox_ros_driver2\n- Add state estimation package/node\n- Add bounding-box-based Offboard\n- For PX4 quad max tilt maneuver, zero the anti-windup gain: const float arw_gain = 2.f / _gain_vel_p(0);\n- ????\n- Profit\n\n### Known Issues\n\n- QGC is started with a virtual joystick (with low throttle if using only VTOLs and centered throttle if there are quads), this is reflective of real-life but note that this counts as \"RC loss\" when switching focus from one autopilot instance to another\n- ArduPilot CIRCLE mode for quads require to explicitly center the virtual throttle with 'rc 3 1500' to keep altitude\n- Gazebo WindEffects plugin is disabled for PX4 standard_vtol\n- Command 178 MAV_CMD_DO_CHANGE_SPEED is accepted but not effective in changing speed for ArduPilot VTOL\n- ArduPilot SITL for Iris uses option -f that also sets \"external\": True, this is not the case for the Alti Transition from ArduPilot/SITL_Models \n- In ArdupilotInterface's action callbacks, std::shared_lock\u003cstd::shared_mutex\u003e lock(node_data_mutex_); could be used on the reads of lat_, lon_, alt_\n- In yolo_inference_node.py, cannot open GPU accelerated (nvh264dec) GStreamer pipeline with cv2.VideoCapture, might need to recompile OpenCV to have both CUDA and GStreamer support (or use python3-gi gir1.2-gst-plugins-base-1.0 gir1.2-gstreamer-1.0 and circumvent OpenCV)\n- QGC does not save roll and pitch in the telemetry bar for PX4 VTOLs (MAV_TYPE 22)\n\n\n\n--\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjacopopan%2Faerial-autonomy-stack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjacopopan%2Faerial-autonomy-stack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjacopopan%2Faerial-autonomy-stack/lists"}