{"id":13442830,"url":"https://github.com/autonomousvision/occupancy_networks","last_synced_at":"2025-04-08T09:08:49.739Z","repository":{"id":37396772,"uuid":"178851471","full_name":"autonomousvision/occupancy_networks","owner":"autonomousvision","description":"This repository contains the code for the paper \"Occupancy Networks - Learning 3D Reconstruction in Function Space\"","archived":false,"fork":false,"pushed_at":"2023-06-27T00:23:54.000Z","size":5299,"stargazers_count":1592,"open_issues_count":82,"forks_count":298,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-04-01T07:52:54.590Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://avg.is.tuebingen.mpg.de/publications/occupancy-networks","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-04-01T11:45:37.000Z","updated_at":"2025-03-28T08:11:43.000Z","dependencies_parsed_at":"2023-10-20T17:58:24.024Z","dependency_job_id":"21a70e35-c211-49fe-be62-f74ea85eb9b8","html_url":"https://github.com/autonomousvision/occupancy_networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/occupancy_networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809962,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:51.823Z","updated_at":"2025-04-08T09:08:49.716Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":["Python",":open_hands: Contributing","单目图像"],"sub_categories":["Project with code"],"readme":"# Occupancy Networks\n![Example 1](img/00.gif)\n![Example 2](img/01.gif)\n![Example 3](img/02.gif)\n\nThis repository contains the code to reproduce the results from the paper\n[Occupancy Networks - Learning 3D Reconstruction in Function Space](https://avg.is.tuebingen.mpg.de/publications/occupancy-networks).\n\nYou can find detailed usage instructions for training your own models and using pretrained models below.\n\nIf you find our code or paper useful, please consider citing\n\n    @inproceedings{Occupancy Networks,\n        title = {Occupancy Networks: Learning 3D Reconstruction in Function Space},\n        author = {Mescheder, Lars and Oechsle, Michael and Niemeyer, Michael and Nowozin, Sebastian and Geiger, Andreas},\n        booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},\n        year = {2019}\n    }\n\n## Installation\nFirst you have to make sure that you have all dependencies in place.\nThe simplest way to do so, is to use [anaconda](https://www.anaconda.com/). \n\nYou can create an anaconda environment called `mesh_funcspace` using\n```\nconda env create -f environment.yaml\nconda activate mesh_funcspace\n```\n\nNext, compile the extension modules.\nYou can do this via\n```\npython setup.py build_ext --inplace\n```\n\nTo compile the dmc extension, you have to have a cuda enabled device set up.\nIf you experience any errors, you can simply comment out the `dmc_*` dependencies in `setup.py`.\nYou should then also comment out the `dmc` imports in `im2mesh/config.py`.\n\n## Demo\n![Example Input](img/example_input.png)\n![Example Output](img/example_output.gif)\n\nYou can now test our code on the provided input images in the `demo` folder.\nTo this end, simply run\n```\npython generate.py configs/demo.yaml\n```\nThis script should create a folder `demo/generation` where the output meshes are stored.\nThe script will copy the inputs into the `demo/generation/inputs` folder and creates the meshes in the `demo/generation/meshes` folder.\nMoreover, the script creates a `demo/generation/vis` folder where both inputs and outputs are copied together.\n\n## Dataset\n\nTo evaluate a pretrained model or train a new model from scratch, you have to obtain the dataset.\nTo this end, there are two options:\n\n1. you can download our preprocessed data\n2. you can download the ShapeNet dataset and run the preprocessing pipeline yourself\n\nTake in mind that running the preprocessing pipeline yourself requires a substantial amount time and space on your hard drive.\nUnless you want to apply our method to a new dataset, we therefore recommmend to use the first option.\n\n### Preprocessed data\nYou can download our preprocessed data (73.4 GB) using\n\n```\nbash scripts/download_data.sh\n```\n\nThis script should download and unpack the data automatically into the `data/ShapeNet` folder.\n\n### Building the dataset\nAlternatively, you can also preprocess the dataset yourself.\nTo this end, you have to follow the following steps:\n* download the [ShapeNet dataset v1](https://www.shapenet.org/) and put into `data/external/ShapeNet`. \n* download the [renderings and voxelizations](http://3d-r2n2.stanford.edu/) from Choy et al. 2016 and unpack them in `data/external/Choy2016` \n* build our modified version of [mesh-fusion](https://github.com/davidstutz/mesh-fusion) by following the instructions in the `external/mesh-fusion` folder\n\nYou are now ready to build the dataset:\n```\ncd scripts\nbash dataset_shapenet/build.sh\n``` \n\nThis command will build the dataset in `data/ShapeNet.build`.\nTo install the dataset, run\n```\nbash dataset_shapenet/install.sh\n```\n\nIf everything worked out, this will copy the dataset into `data/ShapeNet`.\n\n## Usage\nWhen you have installed all binary dependencies and obtained the preprocessed data, you are ready to run our pretrained models and train new models from scratch.\n\n### Generation\nTo generate meshes using a trained model, use\n```\npython generate.py CONFIG.yaml\n```\nwhere you replace `CONFIG.yaml` with the correct config file.\n\nThe easiest way is to use a pretrained model.\nYou can do this by using one of the config files\n```\nconfigs/img/onet_pretrained.yaml\nconfigs/pointcloud/onet_pretrained.yaml\nconfigs/voxels/onet_pretrained.yaml\nconfigs/unconditional/onet_cars_pretrained.yaml\nconfigs/unconditional/onet_airplanes_pretrained.yaml\nconfigs/unconditional/onet_sofas_pretrained.yaml\nconfigs/unconditional/onet_chairs_pretrained.yaml\n```\nwhich correspond to the experiments presented in the paper.\nOur script will automatically download the model checkpoints and run the generation.\nYou can find the outputs in the `out/*/*/pretrained` folders.\n\nPlease note that the config files  `*_pretrained.yaml` are only for generation, not for training new models: when these configs are used for training, the model will be trained from scratch, but during inference our code will still use the pretrained model.\n\n### Evaluation\nFor evaluation of the models, we provide two scripts: `eval.py` and `eval_meshes.py`.\n\nThe main evaluation script is `eval_meshes.py`.\nYou can run it using\n```\npython eval_meshes.py CONFIG.yaml\n```\nThe script takes the meshes generated in the previous step and evaluates them using a standardized protocol.\nThe output will be written to `.pkl`/`.csv` files in the corresponding generation folder which can be processed using [pandas](https://pandas.pydata.org/).\n\nFor a quick evaluation, you can also run\n```\npython eval.py CONFIG.yaml\n```\nThis script will run a fast method specific evaluation to obtain some basic quantities that can be easily computed without extracting the meshes.\nThis evaluation will also be conducted automatically on the validation set during training.\n\nAll results reported in the paper were obtained using the `eval_meshes.py` script.\n\n### Training\nFinally, to train a new network from scratch, run\n```\npython train.py CONFIG.yaml\n```\nwhere you replace `CONFIG.yaml` with the name of the configuration file you want to use.\n\nYou can monitor on \u003chttp://localhost:6006\u003e the training process using [tensorboard](https://www.tensorflow.org/guide/summaries_and_tensorboard):\n```\ncd OUTPUT_DIR\ntensorboard --logdir ./logs --port 6006\n```\nwhere you replace `OUTPUT_DIR` with the respective output directory.\n\nFor available training options, please take a look at `configs/default.yaml`.\n\n# Notes\n* In our paper we used random crops and scaling to augment the input images. \n  However, we later found that this image augmentation decreases performance on the ShapeNet test set.\n  The pretrained model that is loaded in `configs/img/onet_pretrained.yaml` was hence trained without data augmentation and has slightly better performance than the model from the paper. The updated table looks a follows:\n  ![Updated table for single view 3D reconstruction experiment](img/table_img2mesh.png)\n  For completeness, we also provide the trained weights for the model which was used in the paper in  `configs/img/onet_legacy_pretrained.yaml`.\n* Note that training and evaluation of both our model and the baselines is performed with respect to the *watertight models*, but that normalization into the unit cube is performed with respect to the *non-watertight meshes* (to be consistent with the voxelizations from Choy et al.). As a result, the bounding box of the sampled point cloud is usually slightly bigger than the unit cube and may differ a little bit from a point cloud that was sampled from the original ShapeNet mesh.\n\n# Futher Information\nPlease also check out the following concurrent papers that have proposed similar ideas:\n* [Park et al. - DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation (2019)](https://arxiv.org/abs/1901.05103)\n* [Chen et al. - Learning Implicit Fields for Generative Shape Modeling (2019)](https://arxiv.org/abs/1812.02822)\n* [Michalkiewicz et al. - Deep Level Sets: Implicit Surface Representations for 3D Shape Inference (2019)](https://arxiv.org/abs/1901.06802)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Foccupancy_networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Foccupancy_networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Foccupancy_networks/lists"}