{"id":13442827,"url":"https://github.com/autonomousvision/occupancy_flow","last_synced_at":"2025-04-13T08:27:59.816Z","repository":{"id":50309994,"uuid":"215066818","full_name":"autonomousvision/occupancy_flow","owner":"autonomousvision","description":" This repository contains the code for the ICCV 2019 paper \"Occupancy Flow - 4D Reconstruction by Learning Particle Dynamics\"","archived":false,"fork":false,"pushed_at":"2022-10-08T19:00:37.000Z","size":26663,"stargazers_count":221,"open_issues_count":4,"forks_count":29,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-03-27T00:12:05.609Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://avg.is.tuebingen.mpg.de/publications/niemeyer2019iccv","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-14T14:34:05.000Z","updated_at":"2025-03-04T02:29:04.000Z","dependencies_parsed_at":"2023-01-19T15:45:13.018Z","dependency_job_id":null,"html_url":"https://github.com/autonomousvision/occupancy_flow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_flow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_flow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_flow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Foccupancy_flow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/occupancy_flow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248683513,"owners_count":21144919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:51.776Z","updated_at":"2025-04-13T08:27:59.796Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Occupancy Flow \n\u003cdiv style=\"text-align: center\"\u003e\n\u003cimg src=\"animations/field_jumping_jacks.gif\" width=\"256\"/\u003e\n\u003cimg src=\"animations/field_leg_jump.gif\" width=\"256\" /\u003e\n\u003cimg src=\"animations/field_shake_hips.gif\" width=\"256\" /\u003e\n\u003c/div\u003e\n\nThis repository contains the code for the project [Occupancy Flow - 4D Reconstruction by Learning Particle Dynamics](https://avg.is.tuebingen.mpg.de/publications/niemeyer2019iccv).\n\nYou can find detailed usage instructions for training your own models and using pre-trained models below.\n\nIf you find our code or paper useful, please consider citing\n\n    @inproceedings{OccupancyFlow,\n        title = {Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics},\n        author = {Niemeyer, Michael and Mescheder, Lars and Oechsle, Michael and Geiger, Andreas},\n        booktitle = {Proc. of the IEEE International Conf. on Computer Vision (ICCV)},\n        year = {2019}\n    }\n\n## Installation\n### Set up the environment\n#### method 1: local install\nFirst you have to make sure that you have all dependencies in place. The simplest way to do so, is to use [anaconda](https://www.anaconda.com/).\n\nYou can create and activate an anaconda environment called `oflow` using\n\n```\nconda env create -f environment.yaml\nconda activate oflow\n```\n\n#### method 2: using docker\nRun below command. It will set up everything and share the output with host machine.\n```\ndocker compose build\ndocker compose run app bash\n(or in case you have cuda11 compatible gpu) docker compose run cuda11app bash\n```\n\n### Set up Occupancy Flow \nNext, compile the extension modules. You can do this via\n```\npython setup.py build_ext --inplace\n```\n\n## Demo\n\nYou can test our code on the provided input point cloud sequences in the `demo/` folder. To this end, simple run\n```\npython generate.py configs/demo.yaml\n```\nThis script should create a folder `out/demo/` where the output is stored.\n\n## Dataset\n\n### Point-based Data\nTo train a new model from scratch, you have to download the full dataset.\nYou can download the pre-processed data (~42 GB) using\n\n```\nbash scripts/download_data.sh\n```\n\nThe script will download the point-based point-based data for the [Dynamic FAUST (D-FAUST)](http://dfaust.is.tue.mpg.de/) dataset to the `data/` folder. \n\n_Please note_: We do not provide the renderings for the 4D reconstruction from image sequences experiment nor the meshes for the interpolation and generative tasks due to privacy regulations. We outline how you can download the mesh data in the following.\n\n### Mesh Data\n\nPlease follow the instructions on [D-FAUST homepage](http://dfaust.is.tue.mpg.de/) to download the \"female and male registrations\" as well as \"scripts to load / parse the data\".\nNext, follow their instructions in the `scripts/README.txt` file to extract the obj-files of the sequences. Once completed, you should have a folder with the following structure:   \n___\nyour_dfaust_folder/  \n| 50002_chicken_wings/  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| 00000.obj  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| 00001.obj  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| ...  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| 000215.obj  \n| 50002_hips/  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| 00000.obj  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| ...  \n| ...  \n| 50027_shake_shoulders/  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| 00000.obj  \n\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;| ...  \n___\nYou can now run\n```\nbash scripts/migrate_dfaust.sh path/to/your_dfaust_folder\n```\nto copy the mesh data to the dataset folder. \nThe argument has to be the folder to which you have extracted the mesh data (the `your_dfaust_folder` from the directory tree above).\n\n## Usage\n\nWhen you have installed all dependencies and obtained the preprocessed data, you are ready to run our pre-trained models and train new models from scratch.\n\n### Generation\n\n\u003cdiv style=\"text-align: center\"\u003e\n\u003cimg src=\"animations/input_full.gif\" width=\"256\"/\u003e\n\u003cimg src=\"animations/oflow_full.gif\" width=\"256\"/\u003e\n\u003c/div\u003e\n\nTo start the normal mesh generation process using a trained model, use\n\n```\npython generate.py configs/CONFIG.yaml\n```\nwhere you replace `CONFIG.yaml` with the name of the configuration file you want to use.\n\nThe easiest way is to use a pretrained model. You can do this by using one of the config files\n\n```\nconfigs/pointcloud/oflow_w_correspond_pretrained.yaml\nconfigs/interpolation/oflow_pretrained.yaml\nconfigs/generative/oflow_pretrained.yaml\n```\n\nOur script will automatically download the model checkpoints and run the generation. \nYou can find the outputs in the `out/` folder.\n\nPlease note that the config files *_pretrained.yaml are only for generation, not for training new models: when these configs are used for training, the model will be trained from scratch, but during inference our code will still use the pretrained model.\n\n### Generation - Generative Tasks\n\n\u003cdiv style=\"text-align: center\"\u003e\n\u003cimg src=\"animations/interpolation.gif\" width=\"400\"/\u003e\n\u003cimg src=\"animations/motion_transfer.gif\" width=\"400\"/\u003e\n\u003c/div\u003e\n\nFor model-specific latent space interpolations and motion transfers, you first have to run\n```\npython encode_latent_motion_space.py config/generative/CONFIG.yaml\n```\nNext, you can call \n```\npython generate_latent_space_interpolation.py config/generative/CONFIG.yaml\n``` \nor \n```\npython generate_motion_transfer.py config/generative/CONFIG.yaml\n```\n\n*Please note*: Make sure that you use the appropriate model for the generation processes, e.g. the latent space interpolations and motion transfers can only be generated with a generative model (e.g. `configs/generative/oflow_pretrained.yaml`).\n\n### Evaluation\nYou can evaluate the generated output of a model on the test set using\n\n```\npython eval.py configs/CONFIG.yaml\n```\nThe evaluation results will be saved to pickle and csv files.\n\n### Training\n\nFinally, to train a new network from scratch, run\n```\npython train.py configs/CONFIG.yaml\n```\nYou can monitor the training process on http://localhost:6006 using tensorboard:\n```\ncd OUTPUT_DIR\ntensorboard --logdir ./logs --port 6006\n```\nwhere you replace `OUTPUT_DIR` with the respective output directory. For available training options, please have a look at `config/default.yaml`. \n\n\n## Further Information\n\n### Implicit Representations\nIf you like the Occupancy Flow project, please check out our similar projects on inferring [3D shapes (Occupancy Networks)](https://avg.is.tuebingen.mpg.de/publications/occupancy-networks) and [texture (Texture Fields)](https://avg.is.tuebingen.mpg.de/publications/oechsle2019iccv).\n\n### Neural Ordinary  Differential Equations\nIf you enjoyed our approach using differential equations, checkout Ricky Chen et. al.'s awesome implementation of [differentiable ODE solvers](https://github.com/rtqichen/torchdiffeq) which we used in our project.\n\n### Dynamic FAUST Dataset\nWe applied our method to the cool [Dynamic FAUST](http://dfaust.is.tue.mpg.de/) dataset which contains sequences of real humans performing various actions.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Foccupancy_flow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Foccupancy_flow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Foccupancy_flow/lists"}