{"id":18614311,"url":"https://github.com/ChikaYan/d2nerf","last_synced_at":"2025-04-11T00:30:32.468Z","repository":{"id":41278600,"uuid":"494525081","full_name":"ChikaYan/d2nerf","owner":"ChikaYan","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-12T08:45:58.000Z","size":1872,"stargazers_count":189,"open_issues_count":1,"forks_count":14,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-02-22T04:01:40.832Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChikaYan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-20T15:55:56.000Z","updated_at":"2024-12-19T07:39:57.000Z","dependencies_parsed_at":"2024-11-07T03:40:17.504Z","dependency_job_id":null,"html_url":"https://github.com/ChikaYan/d2nerf","commit_stats":null,"previous_names":["d2nerf/d2nerf"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChikaYan%2Fd2nerf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChikaYan%2Fd2nerf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChikaYan%2Fd2nerf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChikaYan%2Fd2nerf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChikaYan","download_url":"https://codeload.github.com/ChikaYan/d2nerf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248322208,"owners_count":21084333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T03:25:54.431Z","updated_at":"2025-04-11T00:30:31.654Z","avatar_url":"https://github.com/ChikaYan.png","language":"Jupyter Notebook","funding_links":[],"categories":["Papers"],"sub_categories":["NeRF"],"readme":"# D\u003csup\u003e2\u003c/sup\u003eNeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video\n\nThis is the code for \"D\u003csup\u003e2\u003c/sup\u003eNeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video\".\n\n![image](imgs/title_card.png)\n\n* [Project Page](https://d2nerf.github.io/): https://d2nerf.github.io/\n\u003c!-- * [Paper](coming): coming --\u003e\n\nThis codebase implements D\u003csup\u003e2\u003c/sup\u003eNeRF based on [HyperNeRF](https://github.com/google/hypernerf) \n\n\n\u003c!-- ## Demo\n\nOur code We provide an easy-to-get-started demo using Google Colab!\n\nThese Colabs will allow you to train a basic version of our method using\nCloud TPUs (or GPUs) on Google Colab.\n\nNote that due to limited compute resources available, these are not the fully\nfeatured models and will train quite slowly and the quality will likely not be that great.\nIf you would like to train a fully featured model, please refer to the instructions below\non how to train on your own machine.\n\n| Description      | Link |\n| ----------- | ----------- |\n| Process a video into a dataset| [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/nerfies/blob/main/notebooks/Nerfies_Capture_Processing.ipynb)|\n| Train HyperNeRF| [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/hypernerf/blob/main/notebooks/HyperNeRF_Training.ipynb)|\n| Render HyperNeRF Videos| [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/google/hypernerf/blob/main/notebooks/HyperNeRF_Render_Video.ipynb)| --\u003e\n\n\n## Setup\nThe code can be run under any environment with Python 3.8 and above.\n\nFirstly, set up an environment via Miniconda or Anaconda:\n\n    conda create --name d2nerf python=3.8\n\nNext, install the required packages:\n\n    pip install -r requirements.txt\n\nInstall the appropriate JAX distribution for your environment by  [following the instructions here](https://github.com/google/jax#installation). For example:\n\n    pip install --upgrade \"jax[cuda]\" -f https://storage.googleapis.com/jax-releases/jax_releases.html\n\n\n## Training\nPlease download our dataset [here](https://drive.google.com/drive/folders/1qm-8P6UqrhimZXp4USzFPumyfu8l1vto?usp=sharing).\n\nAfter unzipping the data, you can train with the following command:\n\n    export DATASET_PATH=/path/to/dataset\n    export EXPERIMENT_PATH=/path/to/save/experiment/to\n    export CONFIG_PATH=configs/rl/001.gin\n    python train.py \\\n        --base_folder $EXPERIMENT_PATH \\\n        --gin_bindings=\"data_dir='$DATASET_PATH'\" \\\n        --gin_configs $CONFIG_PATH\n\nTo plot telemetry to Tensorboard and render checkpoints on the fly, also\nlaunch an evaluation job by running:\n\n    python eval.py \\\n        --base_folder $EXPERIMENT_PATH \\\n        --gin_bindings=\"data_dir='$DATASET_PATH'\" \\\n        --gin_configs $CONFIG_PATH\n\nWe also provide an example script at `train_eval_balloon.sh`.\n\n\n\n## Configuration\n* Similiar to HyperNeRF, We use [Gin](https://github.com/google/gin-config) for configuration.\n* We provide a couple of preset configurations:\n    - `configs/decompose/`: template configurations defining shared comfigurations for NeRF and HyperNeRF\n    - `configs/rl/`: configurations for experiments on real-life scenes. \n    - `configs/synthetic/`: configurations for experiments on synthetic scenes. \n* Please refer to the paper appendix on arxiv to find out the configurations applied to each scene.\n* Please refer to `config.py` for documentation on what each configuration does.\n\n\n\n## Dataset\nThe dataset uses the [same format as Nerfies](https://github.com/google/nerfies#datasets).\n\nFor synthetic scenes generated using [Kubric](https://github.com/google-research/kubric), we also provide the worker script\n, named `script.py` under each folder.\n\n\n\u003c!--\n## Pre-trained Model\nWe include several pre-trained model checkpoints which can be downloaded from [here](https://drive.google.com/file/d/1dlwp52zg9Iik4uKuuumn5V6jbX1tiYeU/view?usp=sharing). Please use the `config.gin` files included in each subfolder for evaluation of the model checkpoints. \n --\u003e\n\n\n\n## Running on own dataset\n\nBecause our code is fully compatiable with HyperNeRF dataset, thanks to them, you can simply use their [colab notebook](https://colab.research.google.com/github/google/nerfies/blob/main/notebooks/Nerfies_Capture_Processing.ipynb) to process your video and prepare a dataset for training.\n\n\n\n\u003c!-- ## Citing\nIf you find our work useful, please consider citing:\n```BibTeX\n@article{park2021hypernerf,\n  author = {Park, Keunhong and Sinha, Utkarsh and Hedman, Peter and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Martin-Brualla, Ricardo and Seitz, Steven M.},\n  title = {HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields},\n  journal = {ACM Trans. Graph.},\n  issue_date = {December 2021},\n  publisher = {ACM},\n  volume = {40},\n  number = {6},\n  month = {dec},\n  year = {2021},\n  articleno = {238},\n}\n``` --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChikaYan%2Fd2nerf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FChikaYan%2Fd2nerf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChikaYan%2Fd2nerf/lists"}