{"id":20620619,"url":"https://github.com/lcav/audioros","last_synced_at":"2025-09-13T21:13:19.797Z","repository":{"id":41157598,"uuid":"286787270","full_name":"LCAV/audioROS","owner":"LCAV","description":"ROS2 stack for audio processing","archived":false,"fork":false,"pushed_at":"2023-06-20T14:25:16.000Z","size":143955,"stargazers_count":30,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-15T12:14:06.523Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LCAV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-11T15:57:34.000Z","updated_at":"2025-04-14T08:53:55.000Z","dependencies_parsed_at":"2023-02-10T01:00:47.719Z","dependency_job_id":null,"html_url":"https://github.com/LCAV/audioROS","commit_stats":null,"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LCAV%2FaudioROS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LCAV%2FaudioROS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LCAV%2FaudioROS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LCAV%2FaudioROS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LCAV","download_url":"https://codeload.github.com/LCAV/audioROS/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249067779,"owners_count":21207396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T12:14:59.652Z","updated_at":"2025-04-15T12:14:17.232Z","avatar_url":"https://github.com/LCAV.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# audioROS\n\n![The Crazylie drone and e-puck2 robot in front of the EPFL logo](doc/epuck-drone.png)\n\nThis repository contains the ROS2-powered stack to do audio processing for sound-based localization on robots. The system was tested on the Crazyflie drone 2.1 and the e-puck2 robot, shown in the image above.\n\n## Overview\n\nThis repository is part of the experimental framework described in the [paper](https://doi.org/10.1109/LRA.2022.3194669):\n```\nF. Dümbgen, A. Hoffet, M. Kolundžija, A. Scholefield, and M. Vetterli, \"Blind as a bat: audible \necholocation on small robots\", IEEE Robotics and Automation Letters (Early Access), 2022.\n```\n\nThe framework includes all components listed below. The components are kept modular so that researchers may focus only on what's relevant to them. For instance, to recreate the audio extension deck and perform more research on audio-based navigation on drones, only repository 2. is necessary; to run the audio-based algorithms in ROS but for a different robot, only repository 1. is enough as a starting point.      \n\n1. [ROS processing pipeline](https://github.com/LCAV/audioROS) (**this repository**) for processing audio for tasks such as obstacle detection and sound source localization. \n2. [Audio deck firmware](https://github.com/LCAV/crazyflie-audio), which also includes the PCB files for reproducing the audio deck. \n3. [Public datasets](https://github.com/LCAV/audio-localization-dataset), for audio-based localization on the e-puck2 robot and the Crazyflie drone. \n4. [Crazyflie firmware](https://github.com/LCAV/crazyflie-firmware) (fork from official vendor firmware), with an added [audio deck driver](https://github.com/LCAV/crazyflie-firmware/blob/master/src/deck/drivers/src/audio_deck.c).\n5. [Gtsam extension](https://github.com/duembgen/gtsam) for performing factor-graph inference using the echolocation measurements. \n\n## Installation\n\nThe repository was built for Ubuntu 20.04 (Focal Fossa) with ROS2 (Galactic) and Python 3.8.12. \n\nTo install, make sure to clone this repo including submodules (e.g. datasets) by running\n```\ngit clone --recurse-submodules https://github.com/LCAV/audioROS\n```\n\nTo install all packages contained in this repo, including dependencies, run (from the root of this repository):\n```\nsudo apt install python3-rosdep python3-colcon-common-extensions\nsudo rosdep init\ncp 19-custom.list /etc/ros/rosdep/sources.list.d/ #might need sudo here\nrosdep update \nrosdep install --from-path src/ --rosdistro $ROS_DISTRO\ncolcon build --symlink-install\n. install/local_setup.bash\n```\n\nUpdate from May 25 2022: because of an inconsistency in matplotlib, in order to run all of the \nanalysis notebooks (using latex rendering), the following two\nnon-standard libraries had to be installed:\n```\nsudo apt install cm-super dvipng\n```\n\n## Contents\n\nThe stack is separated into the following modules:\n\n- `audio_interfaces` (C++): Custom ROS message definintions.\n- `audio_bringup` (python): Pipeline for recording measurements (`measurement_pipeline.py`) and launch files.\n- `audio_gtsam` (python): Classes to build a factor graph from audio and pose measurements.\n- `audio_stack` (python): Read and process audio signals.\n- `audio_simulation` (python): Simulated audio using pyroomacoustics.\n- `audio_publisher` (python): Publish signals from file or computer's audio input stream.\n- `crazyflie_crtp` (python): Publish audio signals received over CRTP from Crazyflie drone.\n- `crazyflie_demo` (python): Demo of drone detecting and avoiding walls.\n- `crazyflie_description` (python): Commonly used global parameters of Crazyflie drone.\n- `topic_plotter` (python): Create plots of the different topics.\n- `topic_writer` (python): Convert data from topics to different formats (e.g. csv format) and save.\n\n\nA diagram of the interactions of the different packages is shown below.\n\n![Diagram showing package interactions](https://app.lucidchart.com/publicSegments/view/8da32e75-dd1a-45f2-a5a3-6a195968585d/image.png)\n\n## Use cases\n\n### Simulation\n\n- To simulate the drone doing wall detection, run:\n  ```\n  ros2 launch audio_bringup wall_simulated.launch.py\n  ```\n- To simulate the drone doing sound source localization inside a room, run:\n  ```\n  ros2 launch audio_bringup doa_simulated.launch.py\n  ```\n  \n### Experiments\n\n- To do real-time wall detection on the Crazyflie drone, run:\n  ```\n  ros2 launch audio_bringup wall_real.launch.py\n  ```\n- To run a new measurement campaign, run:\n  ```\n  ros2 run audio_bringup measurement_pipeline\n  ```\n- To do real-time direction-of-arrival estimation (of an external sound source) on the Crazyflie drone, run:\n  ```\n  ros2 launch audio_bringup doa_real.launch.py\n  ```\n\n### Analysis\n\nThis repo also contains functions to analyze the experimental data, grouped in the folder `python/`. Below table explains where to find what.\n\nNote that the notebooks use preprocessed data to speed up analysis. To process data for the first time, you can use the examples listed in the Makefile. For instance, to preprocess the stepper motor results of the Crazyflie reported in the thesis, run\n`make crayflie_stepper_thesis`. Similarly, for the e-puck results run `make epuck_results`. See `python/Makefile` for all options.\n\nDescriptions:\n\n| notebook            | datasets                 | results                                                  |\n|---------------------|--------------------------|----------------------------------------------------------|\n|  StepperAnalysis    | `2021_07_08_stepper_fast`|   echolocation, distance and frequency slice, matrices [1]  |\n|                     | `2021_07_27_epuck_wall  `|   echolocation, epuck results [1]                           |                                                                                                                                                                                                 \n|  CleanupAnalysis    | `2021_07_08_stepper_fast`|   echolocation, matrix cleaning results [1]                 |\n|  DistanceFlying     | `2021_10_12_flying      `|   echolocation, drone flying towards walls [1]              |\n|                     | `2021_11_23_demo        `|   echolocation, drone avoiding walls [1]                    |\n|                     | `2022_01_27_demo        `|   echolocation, drone avoiding whiteboards [1]              |\n|  ApproachAngleFlying| `2021_05_04_linear      `|   echolocation, approach angle results [2]                  |\n|  WallStudy          | `--                     `|   echolocation, simulation results [1]                      |\n|  DoaAnalysis        | `2021_10_12_doa_stepper `|   doa, experiments on stepper motor [2]                     |\n|                     | `2021_10_12_doa_flying  `|   doa, experiments with hovering drone [2]                  |\n|  DoaStudy           | `--                     `|   doa, simulation results [2]                               |\n\n\n## Credits\n\nWe took inspiration from [this](https://roboticsbackend.com/package-organization-for-a-ros-stack-best-practices/) helpful article for the structure of this repo.\n\n## References\n\nPlease refer to the below publications for more information. \n\n[1] [RA-L paper](https://doi.org/10.1109/LRA.2022.3194669) (main reference)\n```\nF. Dümbgen, A. Hoffet, M. Kolundžija, A. Scholefield, and M. Vetterli, \"Blind as a bat: audible \necholocation on small robots\", IEEE Robotics and Automation Letters (Early Access), 2022. \n```\n\n[2] [Ph.D. dissertation](https://infoscience.epfl.ch/record/290057) (including additional methods and experimental analysis):  \n```\nF. Dümbgen, \"Blind as a bat: spatial perception without sight\", Ph.D. disseration, \nÉcole Polytechnique Fédérale de Lausanne, 2021.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flcav%2Faudioros","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flcav%2Faudioros","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flcav%2Faudioros/lists"}