{"id":17744044,"url":"https://github.com/andreibarsan/dynslam","last_synced_at":"2025-04-04T22:09:11.459Z","repository":{"id":50461599,"uuid":"91897499","full_name":"AndreiBarsan/DynSLAM","owner":"AndreiBarsan","description":"Master's Thesis on Simultaneous Localization and Mapping in dynamic environments. Separately reconstructs both the static environment and the dynamic objects from it, such as cars.","archived":false,"fork":false,"pushed_at":"2021-09-30T19:09:09.000Z","size":21949,"stargazers_count":555,"open_issues_count":19,"forks_count":178,"subscribers_count":23,"default_branch":"master","last_synced_at":"2023-11-07T17:00:42.576Z","etag":null,"topics":["autonomous-vehicles","computer-vision","deep-learning","dense","eth-zurich","master-thesis","slam"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AndreiBarsan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-20T15:18:05.000Z","updated_at":"2023-11-06T07:47:38.000Z","dependencies_parsed_at":"2022-09-05T16:40:44.501Z","dependency_job_id":null,"html_url":"https://github.com/AndreiBarsan/DynSLAM","commit_stats":null,"previous_names":[],"tags_count":4,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreiBarsan%2FDynSLAM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreiBarsan%2FDynSLAM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreiBarsan%2FDynSLAM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AndreiBarsan%2FDynSLAM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AndreiBarsan","download_url":"https://codeload.github.com/AndreiBarsan/DynSLAM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247256115,"owners_count":20909240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-vehicles","computer-vision","deep-learning","dense","eth-zurich","master-thesis","slam"],"created_at":"2024-10-26T06:41:47.661Z","updated_at":"2025-04-04T22:09:11.433Z","avatar_url":"https://github.com/AndreiBarsan.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DynSLAM: Simultaneous Localization and Mapping in Dynamic Environments\n\nThis is a dense SLAM system written in C++. It builds on [InfiniTAM](https://github.com/victorprad/InfiniTAM), adding support\nfor stereo input, outdoor operation, voxel garbage collection,\nand separate dynamic object (e.g., car) reconstruction.\n\nDeveloped as part of my Master's Thesis, in the [Computer\nVision and Geometry Group](https://cvg.ethz.ch) of [ETH\nZurich](https://ethz.ch). Accepted to ICRA 2018 accompanying\nthe paper \"Robust Dense Mapping for Large-Scale Dynamic \nEnvironments\" by Andrei Bârsan, Peidong Liu, Marc Pollefeys, and Andreas Geiger.\n\nThe source code is [hosted on GitHub](https://github.com/AndreiBarsan/DynSLAM).\n\n## Preview\n\nThe following screenshot shows an early preview of DynSLAM in action. It\ntakes in stereo input, computes the depth map, using either ELAS or\ndispnet, segments the input RGB using Multi-task Network Cascades to\ndetect object instances, and then separately reconstructs the static\nbackground and individual object instances.\n\nThe top pane shows the dense reconstruction of the background. The\nfollowing panes show, in top-down, left-right order: the left RGB frame,\nthe computed depth map, the output of the instance-aware semantic\nsegmentation algorithm, the input RGB to the instance reconstructor,\nmemory usage statistics, and a novel view of the reconstructed object\ninstance.\n\nThe colors in the 3D reconstructions correspond to the voxel weights:\nred-tinted areas are low-weight ones, whereas blue ones are high-weight\nones. Areas which remain low-weight even several frames after first\nbeing observed are very likely to be noisy, while blue ones are ones\nwhere the system is confident in its reconstruction.\n\n![DynSLAM GUI screenshot](data/screenshots/dynslam-preview.png)\n\n## Related Repositories\n\n * [My InfiniTAM fork](https://github.com/AndreiBarsan/InfiniTAM), which\n   is used by this system for the actual 3D reconstruction (via\n   volumetric fusion, using voxel hashing for map storage). My fork\n   contains a series of small tweaks designe to make InfiniTAM a little\n   easier to use as a component of a larger system.\n * [My fork of the official implemntation of Multi-task Network Cascades](https://github.com/AndreiBarsan/MNC)\n    for image semantic segmentation. We need this for identifying where\n    the cars are in the input videos. Using semantics enables us to\n    detect both moving and static cars.\n * [My fork of the modified Caffe used by MNC](https://github.com/AndreiBarsan/caffe-mnc). Since MNC's architecture requires\n some tweaks to Caffe's internals, its authors forked Caffe and modified\n it to their needs. I forked their fork and made it work with my tools,\n while also making it faster by merging it with the Caffe master, which\n enabled cuDNN 5 support, among many other things.\n  * [My mirror of libelas](https://github.com/AndreiBarsan/libelas-tooling)\n  which I use for pre-computing the depth maps. I'm working on getting\n  the depth computation to happen on the fly, and investigating other\n  methods for estimating depth from stereo.\n\n## Regenerating Plots\n\nThe plots in the corresponding ICRA paper can all be regenerated from the raw\ndata included in this repository as follows:\n\n  1. Unzip `./raw-data-archives/raw-logz.7z` to `./csv`.\n  1. Install the data analysis dependencies (e.g., in a Python virtual\n     environment or using Anaconda). Installing the pacakges using the Anaconda\n     option can be done as:\n     ```bash\n     conda install --yes jupyter pandas numpy scipy scikit-learn matplotlib seaborn\n     ```\n  1. Start Jupyter:\n     ```bash\n     cd notebooks \u0026\u0026 jupyter notebook\n     ```\n  1. Regenerate Figure 6 using `./notebooks/StaticAndDynamicDepthAnalysis.ipynb`\n  1. Regenerate Figure 7 using `./notebooks/Voxel GC Stats.ipynb`\n  1. The other notebooks can be used to generate the various figures from [the\n     supplementary material](http://andreibarsan.github.io/dynslam).\n\n\n## Building and Running DynSLAM\n\nIf you want to check out the system very quickly, you're in luck!\nThere's a pre-preprocessed sequence you can download to see how it works (see \nthe \"Demo Sequence\" section).\n\nIf you want to preprocess your own sequences, see the \"Preprocessing\" section.\n\n### Building \n\nThis project is built using CMake, and it depends on several submodules. The\nsteps below are designed for Ubuntu 18.04.\nAs such, make sure you don't forget the `--recursive` flag when cloning the \nrepository. If you did\nforget it, just run `git submodule update --init --recursive`.\n\n 1. Clone the repository if you haven't already:\n    ```bash\n    git clone --recursive https://github.com/AndreiBarsan/DynSLAM\n    ```\n 1. Install OpenCV 2.4.9 and CUDA 8.\n 1. [Install docker and nvidia-docker](https://github.com/NVIDIA/nvidia-docker).\n    They are a requirement for preprocessing the data so that it can be consumed\n    by DynSLAM.\n 1. Install the prerequisites (Ubuntu example):\n    ```bash\n    sudo apt-get install libxmu-dev libxi-dev freeglut3 freeglut3-dev libglew-dev glew-utils libpthread-stubs0-dev binutils-dev libgflags-dev libpng++-dev libeigen3-dev\n    ```\n 1. Build Pangolin to make sure it gets put into the CMake registry:\n    ```bash\n    cd src/Pangolin \u0026\u0026 mkdir build/ \u0026\u0026 cd $_ \u0026\u0026 cmake ../ \u0026\u0026 make -j$(nproc)\n    ```\n 1. Build the project in the standard CMake fashion:\n    ```bash\n    mkdir build \u0026\u0026 cd build \u0026\u0026 cmake .. \u0026\u0026 make -j$(nproc)\n    ```\n    \n### Building DynSLAM Inside Docker\n\nWhile the preprocessing makes heavy use of `nvidia-docker` in order to simplify\nthe process, and does so very effectively, running the main DynSLAM program \ninside Docker is still not supported.\n\nThe `Dockerfile` in the root of this project *can* be used to build DynSLAM \ninside a Docker container, but, due to its OpenGL GUI, it cannot run inside it\n(as of February 2018).\n\nSolutions to this problem include using one of the newly released CUDA+OpenGL \nDocker images from NVIDIA as a base image, or fully supporting CLI-only \noperation. Both of these tasks remain part of future work.\n\n\n### Demo Sequence\n 1. After building the project, try processing the demo sequence: \n    [here is a short sample from KITTI Odometry Sequence 06](https://drive.google.com/uc?export=download\u0026confirm=Nnbd\u0026id=1V-I4Tle7MNbmnf2qRe6aTpjxOld2M2i8).\n      1. Extract that to a directory, and run DynSLAM on it (the mkdir circumvents a silly bug):\n        ```bash\n        mkdir -p csv/ \u0026\u0026 build/DynSLAM --use_dispnet --dataset_root=path/to/extracted/archive --dataset_type=kitti-odometry\n        ```\n\n### KITTI Tracking and Odometry Sequences\n 1. The system can run on any KITTI Odometry and Tracking sequence. \n    KITTI Raw sequences should also work, but have not been \n    tested since the evaluation is trickier, as their LIDAR data is not cleaned\n    up to account for the rolling shutter effect. In order to run the system on\n    these sequences, the instance-aware semantic segmentations and dense depth\n    maps must be preprocessed, since DynSLAM does not yet support computing them\n    on the fly. \n    \n    These instructions are for the KITTI Tracking dataset, which is\n    the only one currently supported using helper scripts, but I plan on adding\n    support for easy KITTI Odometry preprocessing, since the only difference\n    between the two datasets is the path structure.\n    1. Use the following download script to grab the KITTI Tracking dataset. Bear in mind\n       that it is a very large dataset which takes up a little over 100Gb of\n       disk space. Sadly, the download is structured such that downloading \n       individual sequences is not possible.\n       ```bash\n       scripts/download_kitti_tracking.py [target-dir]\n       ```\n       By default, this script downloads the data to the `data/kitti/tracking`\n       folder of the DynSLAM project.\n    1. (Alternative) You can also manually grab the KITTI Tracking dataset \n    [from the official website](www.cvlibs.net/datasets/kitti/eval_odometry.php).\n    Make sure you download everything and extract it all in the same directory \n    (see the demo sequence archive for the canonical directory structure, or \n    `Input.h` to see how DynSLAM loads it).\n    1. Preprocess the data using the preprocessing script:\n        ```bash\n        scripts/preprocess-sequence.sh kitti-tracking \u003cdataset-root\u003e \u003ctraining/testing\u003e \u003cnumber\u003e\n        ```\n        For example,\n        ```bash\n        scripts/preprocess-sequence.sh kitti-tracking data/kitti/tracking training 6\n        ```\n        prepares the 6th KITTI Tracking training sequence for DynSLAM.\n    1. Run the pipeline on the KITTI sequence you downloaded.\n       ```bash\n       ./DynSLAM --use_dispnet --enable-evaluation=false --dataset_root=\u003cdataset-root\u003e --dataset_type=kitti-tracking --kitti_tracking_sequence_id=\u003cnumber\u003e\n       ```\n \n You can also use `DynSLAM --help` to view info on additional commandline arguments. (There are a lot of them!)\n\n\n## Citing DynSLAM\n\nIf you use DynSLAM in your research, or build your system off of it, please\nconsider citing the relevant publication:\n\n```\n@INPROCEEDINGS{Barsan2018ICRA,\n  author = {Ioan Andrei B\\^{a}rsan and Peidong Liu and Marc Pollefeys and Andreas Geiger},\n  title = {Robust Dense Mapping for Large-Scale Dynamic Environments},\n  booktitle = {International Conference on Robotics and Automation (ICRA)},\n  year = {2018}\n}\n```\n\n## Remarks\n\n  * The code follows\n    [Google's C++ style guide](https://google.github.io/styleguide/cppguide.html)\n    with the following modifications:\n\n    * The column limit is 100 instead of 80, because of the bias towards\n      longer type names in the code base.\n    * Exceptions are allowed, but must be used judiciously (i.e., for\n      serious errors and exceptional situations).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreibarsan%2Fdynslam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandreibarsan%2Fdynslam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreibarsan%2Fdynslam/lists"}