{"id":21739650,"url":"https://github.com/berkeleyautomation/legs_ros_ws","last_synced_at":"2025-03-21T00:45:21.203Z","repository":{"id":230362793,"uuid":"779168873","full_name":"BerkeleyAutomation/legs_ros_ws","owner":"BerkeleyAutomation","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-18T06:12:45.000Z","size":65391,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"prime","last_synced_at":"2024-11-24T19:42:42.642Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BerkeleyAutomation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-29T07:38:09.000Z","updated_at":"2024-10-18T06:12:49.000Z","dependencies_parsed_at":"2024-10-20T15:50:04.313Z","dependency_job_id":null,"html_url":"https://github.com/BerkeleyAutomation/legs_ros_ws","commit_stats":null,"previous_names":["berkeleyautomation/legs_ros_ws"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerkeleyAutomation%2Flegs_ros_ws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerkeleyAutomation%2Flegs_ros_ws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerkeleyAutomation%2Flegs_ros_ws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BerkeleyAutomation%2Flegs_ros_ws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BerkeleyAutomation","download_url":"https://codeload.github.com/BerkeleyAutomation/legs_ros_ws/tar.gz/refs/heads/prime","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244717336,"owners_count":20498283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-26T06:09:51.171Z","updated_at":"2025-03-21T00:45:21.177Z","avatar_url":"https://github.com/BerkeleyAutomation.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LEGS\nHi! Welcome to the ROS2 workspace setup for LEGS. This README details how to setup the ROS2 environment with Droid SLAM and how to collect data and get Image-Pose correspondances for our hardware setup with 2 left and right facing Zed2s and a front facing Realsense D455. However, hopefully things should be clear enough to adapt to whatever depth camera hardware you are using. Hopefully, you can read more about this in our IROS 2024 paper linked below\n\n## Prerequesites\nComputer with Ubuntu 22.04 and GPU with at least 20 GB RAM (this is where we have observed the max spike occurs). If you have a smaller GPU, things should still work, you just have to downsample your images, which could mitigate quality but it shouldn't significantly make things worse.\n\n## Installation (Bash Script)\n\nClone the repo:\n```\ncd ~/\nmkdir legs_ws\ncd legs_ws\ngit clone --recurse-submodules https://github.com/BerkeleyAutomation/legs_ros_ws.git src\n```\n\nRun the setup bash script\n```\ncd ~/legs_ws/src\nbash env_setup.bash\n```\n\n## Installation (Step by Step)\n\n### Step 1: Install ROS2 Humble Full: https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html\n```\n# Step 1.1 Set Locale\nsudo apt update -y\nsudo apt install locales\nsudo locale-gen en_US en_US.UTF-8\nsudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8\nexport LANG=en_US.UTF-8\n\n# Step 1.2 Setup Sources\nsudo apt install software-properties-common\necho -ne '\\n' | sudo add-apt-repository universe\nsudo apt update \u0026\u0026 sudo apt install curl -y\nsudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg\necho \"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release \u0026\u0026 echo $UBUNTU_CODENAME) main\" | sudo tee /etc/apt/sources.list.d/ros2.list \u003e /dev/null\n\n# Step 1.3 Install ROS2 Packages\nsudo apt update -y\nsudo apt upgrade -y\nsudo apt install ros-humble-desktop-full -y\necho \"source /opt/ros/humble/setup.bash\" \u003e\u003e ~/.bashrc\n```\n\n### Step 2: Install Mamba: https://robofoundry.medium.com/using-robostack-for-ros2-9bb52ca89c12\n```\ncd ~/\ncurl -L -O \"https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh\"\n(echo -e \"\\nyes\"; echo -e \"\\nyes\\n\")| bash Mambaforge-$(uname)-$(uname -m).sh\nsource ~/.bashrc\nconda install mamba -c conda-forge -y\n```\n\n### Step 3: Create Mamba environment with nerfstudio: https://docs.nerf.studio/quickstart/installation.html\n```\nmamba create -n droid_slam_ros_env python=3.10.12 -y\nmamba activate droid_slam_ros_env\npython -m pip install --upgrade pip\npip uninstall torch torchvision functorch tinycudann\npip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118\nconda install -c \"nvidia/label/cuda-11.8.0\" cuda-toolkit -y\npip install setuptools==69.5.1 # ImportError: cannot import name packaging from pkg_resources\npip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch\npip install nerfstudio\n```\n\n### Step 4: Setup ROS2 Humble with Mamba environment: https://robofoundry.medium.com/using-robostack-for-ros2-9bb52ca89c12 and https://robostack.github.io/GettingStarted.html\n```\nconda config --env --add channels conda-forge\nconda config --env --add channels robostack\nconda config --env --add channels robostack-humble\nconda config --env --add channels robostack-experimental\nmamba install ros-humble-desktop-full # Will fail\nconda config --env --add channels conda-forge\nconda config --env --add channels robostack-staging\nconda config --env --remove channels defaults\nmamba install ros-humble-desktop -y\nmamba install ros-humble-desktop-full -y\npip install torch-scatter==2.1.1\npip install matplotlib==3.7.2\npip install matplotlib-inline==0.1.6\n```\n\n### Step 5: Install Realsense2 Package\n```\necho \"source /opt/ros/humble/setup.bash\" \u003e\u003e ~/.bashrc\nsudo apt-get install ros-humble-realsense2-camera -y\n```\n\n### Step 6: Install Zed SDK: https://www.stereolabs.com/developers/release\n```\ncd ~/\nwget https://download.stereolabs.com/zedsdk/4.1/cu121/ubuntu22\nchmod +x ubuntu22\n./ubuntu22 # Install everything with default settings (say yes) except for the optimize models because we didn't want to wait for it to take multiple hours\n```\n\n### Step 7: Install Zed ROS2 Package: https://www.stereolabs.com/docs/ros2\n```\ncd ~/legs_ws\nrosdep install --from-paths src --ignore-src -r -y\ncolcon build --cmake-args=-DCMAKE_BUILD_TYPE=Release\n```\n\n### Step 8: Install OpenCV and OpenCV Contrib: https://docs.opencv.org/4.x/d7/d9f/tutorial_linux_install.html\n```\ncd ~/\nsudo apt update \u0026\u0026 sudo apt install -y cmake g++ wget unzip\nwget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip\nwget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.x.zip\nunzip opencv.zip\nunzip opencv_contrib.zip\nmkdir -p build \u0026\u0026 cd build\ncmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x\ncmake --build .\n```\n### Step 9: Install ROS2 Image Transport\n```\nsudo apt-get install ros-humble-image-transport\n```\n\n### Step 10: Build workspace\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\n```\n\n## Run Project\n\n### Run Realsense D455\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 launch camera_bringup realsense_d455.launch.py\n```\n\nTo verify things are working, you can open Rviz and view topics /camera/color/image_raw and /camera/depth/image_rect_raw.\n\n### Run Left Zed (with TRI Stereo Model: https://arxiv.org/pdf/2109.11644)\n\nTerminal 1\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 launch zed_wrapper left_zed_camera.launch.py\n```\nTerminal 2\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run tri_model_ros_wrapper left_tri_model_node.py\n```\nTo verify things are working, you can open Rviz and view topics /tri_left_zed_cropped and /tri_left_zed_depth. If you don't have access to the super secret, level 13 classified clearance stereo model, you should be able to use topics /left_zed/zed_node/left/image_rect_color and /left_zed/zed_node/depth/depth_registered.\n\n### Run Right Zed (with TRI Stereo Model: https://arxiv.org/pdf/2109.11644)\n\nTerminal 1\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 launch zed_wrapper right_zed_camera.launch.py\n```\nTerminal 2\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run tri_model_ros_wrapper right_tri_model_node.py\n```\nTo verify things are working, you can open Rviz and view topics /tri_right_zed_cropped and /tri_right_zed_depth. If you don't have access to the super secret, level 13 classified clearance stereo model, you should be able to use topics /right_zed/zed_node/left/image_rect_color and /right_zed/zed_node/depth/depth_registered.\n\n### Run all cameras synced up together\nRun the Realsense D455, left Zed, and right Zed as explained above. Then, also open a terminal and run the following. This node will make sure all the timestamps are synced up, and then publish all the images at 5 Hz to reduce the strain on network bandwidth. This frame rate can be changed within the node.\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run camera_bringup synced_image_transport_node\n```\nTo verify things are working, see if the topics are being published at the proper frequency. It may be tough to view the images on Rviz traditionally, but you can run the uncompressed_zed_realsense_image_transport_node node to visualize the compressed images.\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run camera_bringup uncompressed_zed_realsense_image_transport_node\n```\n\n### Save synced camera data to onboard computer\nRun the Realsense D455, left Zed, and right Zed as explained above. Then, also open a terminal and run the following. This node will save all the RGB and depth images on the computer as pngs and numpy files.\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run camera_bringup auto_splat_sync_collector_node.py\n```\nTo verify things are working, check to see if all images are saved in the folder called sync_images.\n\n### Convert synced camera data to ROS2 bag\nSo let's say you made a bunch of RGB and depth folders from the auto_splat_sync_collector_node. If we want to publish these files to ROS2 topics and make a ROS2 bag out of it, run the following 2 terminals. Before you run the convert_d455_2_zed_folder_to_bag.py script, change the d455_folder variable to the desired filepath.\n\nTerminal 1\n```\nros2 bag record -o data_bag -a\n```\n\nTerminal 2\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run droid_slam_ros convert_d455_2_zed_folder_to_bag.py\n```\nTo verify things are working, run the view_multi_compressed_bag.py node which should view all the images.\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run droid_slam_ros view_multi_compressed_bag.py\n```\n\n### Run DROID-SLAM with spacebar bundle adjust\nEither play a data bag or run the cameras all synced up together and then run the following on 2 terminals. \n\nTerminal 1\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run droid_slam_ros space_bundle_adjust_node.py\n```\n\nTerminal 2\n```\ncd ~/legs_ws\ncolcon build\n. install/setup.bash\nros2 run droid_slam_ros multi_ptcloud_d455_2zed_global_real_droid_subscriber_node_prime.py\n```\nTo verify things are working, you should be able to open a Viser window in the browser and see the SLAM happening in realtime. Everytime you want to run the global bundle adjustment, make sure the robot isn't moving and hit the spacebar in the bundle adjust node terminal.\n\n## Errors\nIf you try to build the ROS2 workspace and see this error: CMake Error at /opt/ros/humble/share/rosidl_typesupport_c/cmake/get_used_typesupports.cmake:35 (message): No 'rosidl_typesupport_c' found.\nMake soure you have \"source /opt/ros/humble/setup.bash\" in your .bashrc. You can also try \"sudo apt-get install ros-humble-rosidl-typesupport-c\".\n\n## Bibtex\nIf you find this useful, please cite the paper!\n\u003cpre id=\"codecell0\"\u003e@article{yu2024language,\n\u0026nbsp;author = {Yu, Justin and Hari, Kush and Srinivas, Kishore and El-Refai, Karim and Rashid, Adam and Kim, Chung Min and Kerr, Justin and Cheng, Richard and Irshad, Muhammad Zubair and Balakrishna, Ashwin and Kollar, Thomas and Goldberg, Ken},\n\u0026nbsp;title = {Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot},\n\u0026nbsp;booktitle = {International Conference on Intelligent Robots and Systems (IROS)},\n\u0026nbsp;year = {2024},\n} \u003c/pre\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fberkeleyautomation%2Flegs_ros_ws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fberkeleyautomation%2Flegs_ros_ws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fberkeleyautomation%2Flegs_ros_ws/lists"}