{"id":18600795,"url":"https://github.com/autonomousvision/convolutional_occupancy_networks","last_synced_at":"2025-04-12T23:40:47.960Z","repository":{"id":37395916,"uuid":"283316497","full_name":"autonomousvision/convolutional_occupancy_networks","owner":"autonomousvision","description":"[ECCV'20] Convolutional Occupancy Networks","archived":false,"fork":false,"pushed_at":"2023-07-31T20:30:37.000Z","size":11005,"stargazers_count":869,"open_issues_count":13,"forks_count":119,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-04-12T23:40:13.952Z","etag":null,"topics":["3d-deep-learning","3d-reconstruction","eccv-2020","implicit-representions","neural-fields","point-clouds","scene-representations","surface-reconstruction"],"latest_commit_sha":null,"homepage":"https://pengsongyou.github.io/conv_onet","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-28T20:09:22.000Z","updated_at":"2025-04-11T01:03:08.000Z","dependencies_parsed_at":"2024-12-03T23:02:23.514Z","dependency_job_id":"923f8a8d-8e84-40f0-9876-37e979e5189c","html_url":"https://github.com/autonomousvision/convolutional_occupancy_networks","commit_stats":{"total_commits":17,"total_committers":2,"mean_commits":8.5,"dds":0.05882352941176472,"last_synced_commit":"838bea5b2f1314f2edbb68d05ebb0db49f1f3bd2"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fconvolutional_occupancy_networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fconvolutional_occupancy_networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fconvolutional_occupancy_networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fconvolutional_occupancy_networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/convolutional_occupancy_networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248647254,"owners_count":21139081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-deep-learning","3d-reconstruction","eccv-2020","implicit-representions","neural-fields","point-clouds","scene-representations","surface-reconstruction"],"created_at":"2024-11-07T02:05:37.041Z","updated_at":"2025-04-12T23:40:47.923Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Convolutional Occupancy Networks\n[**Paper**](https://arxiv.org/pdf/2003.04618.pdf) | [**Supplementary**](http://www.cvlibs.net/publications/Peng2020ECCV_supplementary.pdf) | [**Video**](https://www.youtube.com/watch?v=EmauovgrDSM) | [**Teaser Video**](https://youtu.be/k0monzIcjUo) | [**Project Page**](https://pengsongyou.github.io/conv_onet) | [**Blog Post**](https://autonomousvision.github.io/convolutional-occupancy-networks/) \u003cbr\u003e\n\n\u003cdiv style=\"text-align: center\"\u003e\n\u003cimg src=\"media/teaser_matterport.gif\" width=\"600\"/\u003e\n\u003c/div\u003e\n\nThis repository contains the implementation of the paper:\n\nConvolutional Occupancy Networks  \n[Songyou Peng](https://pengsongyou.github.io/), [Michael Niemeyer](https://m-niemeyer.github.io/), [Lars Mescheder](https://is.tuebingen.mpg.de/person/lmescheder), [Marc Pollefeys](https://www.inf.ethz.ch/personal/pomarc/) and [Andreas Geiger](http://www.cvlibs.net/)  \n**ECCV 2020 (spotlight)**  \n\nIf you find our code or paper useful, please consider citing\n```bibtex\n@inproceedings{Peng2020ECCV,\n author =  {Peng, Songyou and Niemeyer, Michael and Mescheder, Lars and Pollefeys, Marc and Geiger, Andreas},\n title = {Convolutional Occupancy Networks},\n booktitle = {European Conference on Computer Vision (ECCV)},\n year = {2020}}\n```\nContact [Songyou Peng](mailto:songyou.pp@gmail.com) for questions, comments and reporting bugs.\n\n\n## Installation\nFirst you have to make sure that you have all dependencies in place.\nThe simplest way to do so, is to use [anaconda](https://www.anaconda.com/). \n\nYou can create an anaconda environment called `conv_onet` using\n```\nconda env create -f environment.yaml\nconda activate conv_onet\n```\n**Note**: you might need to install **torch-scatter** mannually following [the official instruction](https://github.com/rusty1s/pytorch_scatter#pytorch-140):\n```\npip install torch-scatter==2.0.4 -f https://pytorch-geometric.com/whl/torch-1.4.0+cu101.html\n```\n\nNext, compile the extension modules.\nYou can do this via\n```\npython setup.py build_ext --inplace\n```\n\n## Demo\nFirst, run the script to get the demo data:\n```\nbash scripts/download_demo_data.sh\n```\n### Reconstruct Large-Scale Matterport3D Scene\nYou can now quickly test our code on the real-world scene shown in the teaser. To this end, simply run:\n```\npython generate.py configs/pointcloud_crop/demo_matterport.yaml\n```\nThis script should create a folder `out/demo_matterport/generation` where the output meshes and input point cloud are stored.\n\n**Note**: This experiment corresponds to our **fully convolutional model**, which we train only on the small crops from our synthetic room dataset. This model can be directly applied to large-scale real-world scenes with real units and generate meshes in a sliding-window manner, as shown in the [teaser](media/teaser_matterport.gif). More details can be found in section 6 of our [supplementary material](http://www.cvlibs.net/publications/Peng2020ECCV_supplementary.pdf). For training, you can use the script `pointcloud_crop/room_grid64.yaml`.\n\n\n### Reconstruct Synthetic Indoor Scene\n\u003cdiv style=\"text-align: center\"\u003e\n\u003cimg src=\"media/demo_syn_room.gif\" width=\"600\"/\u003e\n\u003c/div\u003e\n\nYou can also test on our synthetic room dataset by running: \n```\npython generate.py configs/pointcloud/demo_syn_room.yaml\n```\n## Dataset\n\nTo evaluate a pretrained model or train a new model from scratch, you have to obtain the respective dataset.\nIn this paper, we consider 4 different datasets:\n\n### ShapeNet\nYou can download the dataset (73.4 GB) by running the [script](https://github.com/autonomousvision/occupancy_networks#preprocessed-data) from Occupancy Networks. After, you should have the dataset in `data/ShapeNet` folder.\n\n### Synthetic Indoor Scene Dataset\nFor scene-level reconstruction, we create a synthetic dataset of 5000\nscenes with multiple objects from ShapeNet (chair, sofa, lamp, cabinet, table). There are also ground planes and randomly sampled walls.\n\nYou can download our preprocessed data (144 GB) using\n\n```\nbash scripts/download_data.sh\n```\n\nThis script should download and unpack the data automatically into the `data/synthetic_room_dataset` folder.  \n**Note**: We also provide **point-wise semantic labels** in the dataset, which might be useful.\n\n\nAlternatively, you can also preprocess the dataset yourself.\nTo this end, you can:\n* download the ShapeNet dataset as described above.\n* check `scripts/dataset_synthetic_room/build_dataset.py`, modify the path and run the code.\n\n### Matterport3D\nDownload Matterport3D dataset from [the official website](https://niessner.github.io/Matterport/). And then, use `scripts/dataset_matterport/build_dataset.py` to preprocess one of your favorite scenes. Put the processed data into `data/Matterport3D_processed` folder.\n\n### ScanNet\nDownload ScanNet v2 data from the [official ScanNet website](https://github.com/ScanNet/ScanNet).\nThen, you can preprocess data with:\n`scripts/dataset_scannet/build_dataset.py` and put into `data/ScanNet` folder.  \n**Note**: Currently, the preprocess script normalizes ScanNet data to a unit cube for the comparison shown in the paper, but you can easily adapt the code to produce data with real-world metric. You can then use our fully convolutional model to run evaluation in a sliding-window manner.\n\n## Usage\nWhen you have installed all binary dependencies and obtained the preprocessed data, you are ready to run our pre-trained models and train new models from scratch.\n\n### Mesh Generation\nTo generate meshes using a trained model, use\n```\npython generate.py CONFIG.yaml\n```\nwhere you replace `CONFIG.yaml` with the correct config file.\n\n**Use a pre-trained model**  \nThe easiest way is to use a pre-trained model. You can do this by using one of the config files under the `pretrained` folders.\n\nFor example, for 3D reconstruction from noisy point cloud with our 3-plane model on the synthetic room dataset, you can simply run:\n```\npython generate.py configs/pointcloud/pretrained/room_3plane.yaml\n```\nThe script will automatically download the pretrained model and run the generation. You can find the outputs in the `out/.../generation_pretrained` folders\n\nNote that the config files are only for generation, not for training new models: when these configs are used for training, the model will be trained from scratch, but during inference our code will still use the pretrained model.\n\n\nWe provide the following pretrained models:\n```\npointcloud/shapenet_1plane.pt\npointcloud/shapenet_3plane.pt\npointcloud/shapenet_grid32.pt\npointcloud/shapenet_3plane_partial.pt\npointcloud/shapenet_pointconv.pt\npointcloud/room_1plane.pt\npointcloud/room_3plane.pt\npointcloud/room_grid32.pt\npointcloud/room_grid64.pt\npointcloud/room_combine.pt\npointcloud/room_pointconv.pt\npointcloud_crop/room_grid64.pt\nvoxel/voxel_shapenet_1plane.pt\nvoxel/voxel_shapenet_3plane.pt\nvoxel/voxel_shapenet_grid32.pt\n```\n\n### Evaluation\nFor evaluation of the models, we provide the script `eval_meshes.py`. You can run it using:\n```\npython eval_meshes.py CONFIG.yaml\n```\nThe script takes the meshes generated in the previous step and evaluates them using a standardized protocol. The output will be written to `.pkl/.csv` files in the corresponding generation folder which can be processed using [pandas](https://pandas.pydata.org/).\n\n**Note:** We follow previous works to use \"use 1/10 times the maximal edge length of the current object’s bounding box as unit 1\" (see [Section 4 - Metrics](http://www.cvlibs.net/publications/Mescheder2019CVPR.pdf)). In practice, this means that we multiply the Chamfer-L1 by a factor of 10 for reporting the numbers in the paper.\n\n### Training\nFinally, to train a new network from scratch, run:\n```\npython train.py CONFIG.yaml\n```\nFor available training options, please take a look at `configs/default.yaml`.\n\n## Further Information\nPlease also check out the following concurrent works that either tackle similar problems or share similar ideas:\n- [[CVPR 2020] Jiang et al. - Local Implicit Grid Representations for 3D Scenes](https://arxiv.org/abs/2003.08981)\n- [[CVPR 2020] Chibane et al. Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://arxiv.org/abs/2003.01456)\n- [[ECCV 2020] Chabra et al. - Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://arxiv.org/abs/2003.10983)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Fconvolutional_occupancy_networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Fconvolutional_occupancy_networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Fconvolutional_occupancy_networks/lists"}