{"id":13723168,"url":"https://github.com/magicleap/Atlas","last_synced_at":"2025-05-07T16:32:15.038Z","repository":{"id":39645049,"uuid":"279325121","full_name":"magicleap/Atlas","owner":"magicleap","description":"Atlas: End-to-End 3D Scene Reconstruction from Posed Images","archived":false,"fork":false,"pushed_at":"2022-04-06T16:02:35.000Z","size":1425,"stargazers_count":1834,"open_issues_count":52,"forks_count":221,"subscribers_count":50,"default_branch":"master","last_synced_at":"2025-04-08T09:06:48.463Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/magicleap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-13T14:26:30.000Z","updated_at":"2025-04-02T09:02:31.000Z","dependencies_parsed_at":"2022-07-22T05:02:24.167Z","dependency_job_id":null,"html_url":"https://github.com/magicleap/Atlas","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicleap%2FAtlas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicleap%2FAtlas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicleap%2FAtlas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicleap%2FAtlas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/magicleap","download_url":"https://codeload.github.com/magicleap/Atlas/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252915436,"owners_count":21824559,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T01:01:36.816Z","updated_at":"2025-05-07T16:32:10.024Z","avatar_url":"https://github.com/magicleap.png","language":"Python","funding_links":[],"categories":["Simultaneous Localization and Mapping","Python"],"sub_categories":["Visual"],"readme":"# ATLAS: End-to-End 3D Scene Reconstruction from Posed Images\n\n### [Project Page](http://zak.murez.com/atlas) | [Paper](https://arxiv.org/abs/2003.10432) | [Video](https://youtu.be/9NOPcOGV6nU) | [Models](https://drive.google.com/file/d/12P29x6revvNWREdZ01ufJwMFPl-FEI_V/view?usp=sharing) | [Sample Data](https://drive.google.com/file/d/13-D7QNVZjj864E768zJ7IWDDqY6agUES/view?usp=sharing)\n\n[Zak Murez](http://zak.murez.com), \n[Tarrence van As](https://github.com/tarrencev),\n[James Bartolozzi](http://jhb.graphics),\nAyan Sinha,\nVijay Badrinarayanan, and \nAndrew Rabinovich\n\n\u003cimg src='imgs/AtlasGIF.gif'/\u003e\n\u003cimg src='imgs/figure1.jpg'/\u003e\n\n## Quickstart\nWe provide a [Colab Notebook](https://colab.research.google.com/drive/19_kJSkrQqPhQGIWpEpbZlk48b8p5ZRTO?usp=sharing) to try inference.\n\n## Installation\nWe provide a docker image `Docker/Dockerfile` with all the dependencies.\n\nOr you can install them yourself:\n```\nconda install -y pytorch=1.5.0 torchvision=0.6.0 cudatoolkit=10.2 -c pytorch\nconda install opencv\npip install \\\n  open3d\u003e=0.10.0.0 \\\n  trimesh\u003e=3.7.6 \\\n  pyquaternion\u003e=0.9.5 \\\n  pytorch-lightning\u003e=0.8.5 \\\n  pyrender\u003e=0.1.43\npython -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.5/index.html\n```\nFor 16bit mixed precision (default training setting) you will also need [NVIDIA apex](https://github.com/NVIDIA/apex)\n```\ngit clone https://github.com/NVIDIA/apex\npip install -v --no-cache-dir --global-option=\"--cpp_ext\" --global-option=\"--cuda_ext\" ./apex\n```\n\nFor headless rendering with pyrender (used for evaluation) see installation instructions [here](https://pyrender.readthedocs.io/en/latest/install/index.html).\n\nFor inference with COLMAP see installation instructions [here](https://colmap.github.io/).\n\n(If you have problems running the code try using the exact versions specified... for example the pytorch-lightning API has not settled yet).\n\n## Data Preperation\n#### Sample\nWe provide a small [sample scene](https://drive.google.com/file/d/13-D7QNVZjj864E768zJ7IWDDqY6agUES/view?usp=sharing) for easy download and rapid inference.\nDownload and extract the data to `DATAROOT`. The directory structure should look like:\n```\nDATAROOT\n└───sample\n│   └───sample1\n│       │   intrinsics.txt\n│       └───color\n│       │   │   00000001.jpg\n│       │   │   00000002.jpg\n│       │   │   ...\n│       └───pose\n│           │   00000001.txt\n│           │   00000002.txt\n│           │   ...\n```\nNext run our data preperation script which parses the raw data format into our common json format (more info [here](atlas/datasets/README.md))\n(note that we store our derivered data in a seperate folder `METAROOT` to prevent pollution of the original data).\n```\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset sample\n```\n\n#### Scannet\nDownload and extract Scannet by following the instructions provided at http://www.scan-net.org/.\nYou also need to download the train/val/test splits and the label mapping from https://github.com/ScanNet/ScanNet (Benchmark Tasks).\nThe directory structure should look like:\n```\nDATAROOT\n└───scannet\n│   └───scans\n│   |   └───scene0000_00\n│   |       └───color\n│   |       │   │   0.jpg\n│   |       │   │   1.jpg\n│   |       │   │   ...\n│   |       │   ...\n│   └───scans_test\n│   |       └───color\n│   |       │   │   0.jpg\n│   |       │   │   1.jpg\n│   |       │   │   ...\n│   |       │   ...\n|   └───scannetv2-labels.combined.tsv\n|   └───scannetv2_test.txt\n|   └───scannetv2_train.txt\n|   └───scannetv2_val.txt\n```\nNext run our data preperation script which parses the raw data format into our common json format (more info [here](atlas/datasets/README.md))\n(note that we store our derivered data in a seperate folder `METAROOT` to prevent pollution of the original data).\nThis script also generates the ground truth TSDFs using TSDF Fusion.\n```\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset scannet\n```\nThis will take a while (a couple hours on 8 Quadro RTX 6000's)... if you have multiple gpus you can use the `--i` and `--n` flags to run in parallel\n```\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset scannet --i 0 --n 4 \u0026\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset scannet --i 1 --n 4 \u0026\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset scannet --i 2 --n 4 \u0026\npython prepare_data.py --path DATAROOT --path_meta METAROOT --dataset scannet --i 3 --n 4 \u0026\n```\nNote that if you do not plan to train you can prepare just the test set using the `--test` flag.\n\n#### Your own data\nTo use your own data you will need to put it in the same format as the sample data, or implement your own version of something like [sample.py](atlas/datasets/sample.py). After that you can modify `prepare_data.py` to also prepare your data.\nNote that the pretrained models are trained with Z-up metric coordinates and do not generalize to other coordinates (this means that the scale and 2 axes of the orientation ambiguity of SFM must be resolved prior to using the poses).\n\n## Inference\nOnce you have downloaded and prepared the data (as described above) you can run inference using our pretrained model ([download](https://drive.google.com/file/d/12P29x6revvNWREdZ01ufJwMFPl-FEI_V/view?usp=sharing)) or by training your own (see below).\n\nTo run on the sample scene use:\n```\npython inference.py --model results/release/semseg/final.ckpt --scenes METAROOT/sample/sample1/info.json\n```\nIf your GPU does not have enough memory you can reduce `voxel_dim` (at the cost of possible clipping the scene)\n```\npython inference.py --model results/release/semseg/final.ckpt --scenes METAROOT/sample/sample1/info.json --voxel_dim 208 208 80\n```\nNote that the values of voxel_dim must be divisible by 8 using the default 3D network.\n\nResults will be saved to:\n```\nresults/release/semseg/test_final/sample1.ply // mesh\nresults/release/semseg/test_final/sample1.npz // tsdf\nresults/release/semseg/test_final/sample1_attributes.npz // vertex semseg\n```\n\nTo run on the entire Scannet test set use:\n```\npython inference.py --model results/release/semseg/final.ckpt\n```\n\n## Evaluation\nAfter running inference on Scannet you can run evaluation using:\n```\npython evaluate.py --model results/release/semseg/test_final/\n```\n\nNote that `evaluate.py` uses pyrender to render depth maps from the predicted mesh for 2D evaluation.\nIf you are using headless rendering you must also set the enviroment variable `PYOPENGL_PLATFORM=osmesa`\n(see [pyrender](https://pyrender.readthedocs.io/en/latest/install/index.html) for more details).\n\nYou can print the results of a previous evaluation run using\n```\npython visualize_metrics.py --model results/release/semseg/test_final/\n```\n\n\n## Training\nIn addition to downloadinng and prepareing the data (as described above) you will also need to [download](https://drive.google.com/file/d/15x8k-YOs_65N35CJafoJAPiftyX4Lx5w/view?usp=sharing) our pretrained resnet50 weights (ported from [detectron2](https://github.com/facebookresearch/detectron2)) and unnzip it.\n\nThen you can train your own models using `train.py`.\n\nConfiguration is controlled via a mix of config.yaml files and command line arguments.\nWe provide a few sample config files used in the paper in `configs/`.\nExperiment names are specified by `TRAINER.NAME` and `TRAINER.VERSION`, which default to `atlas` and `default`.\nSee config.py for a full list of parameters.\n\n```\npython train.py --config configs/base.yaml TRAINER.NAME atlas TRAINER.VERSION base\n```\n```\npython train.py --config configs/semseg.yaml TRAINER.NAME atlas TRAINER.VERSION semseg\n```\n\nTo watch training progress use\n```\ntensorboard --logdir results/\n```\n\n## COLMAP Baseline\nWe also provide scripts to run inference and evaluataion using COLMAP. Note that you must install [COLMAP](https://colmap.github.io/) (which is included in our docker image).\n\nFor inference on the sample scene use\n```\npython inference_colmap.py --pathout results/colmap --scenes METAROOT/sample/sample1/info.json\n```\nand for Scannet\n```\npython inference_colmap.py --pathout results/colmap\n```\n\nTo evaluate Scannet use\n```\npython evaluate_colmap.py --pathout results/colmap\n```\n\n## Citation\n\n```\n@inproceedings{murez2020atlas,\n  title={Atlas: End-to-End 3D Scene Reconstruction from Posed Images},\n  author={Zak Murez and \n          Tarrence van As and \n          James Bartolozzi and \n          Ayan Sinha and \n          Vijay Badrinarayanan and \n          Andrew Rabinovich},\n  booktitle = {ECCV},\n  year      = {2020},\n  url       = {https://arxiv.org/abs/2003.10432}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicleap%2FAtlas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmagicleap%2FAtlas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicleap%2FAtlas/lists"}