{"id":18600793,"url":"https://github.com/autonomousvision/factor-fields","last_synced_at":"2025-07-27T21:37:11.709Z","repository":{"id":183924537,"uuid":"633461808","full_name":"autonomousvision/factor-fields","owner":"autonomousvision","description":"[SIGGRAPH 2023] We provide a unified formula for neural fields (Factor Fields) and a novel dictionary factorization (Dictionary Fields)","archived":false,"fork":false,"pushed_at":"2024-10-22T11:30:02.000Z","size":5429,"stargazers_count":213,"open_issues_count":6,"forks_count":11,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-11T04:08:15.484Z","etag":null,"topics":["dictionary","nerf","representation-learning"],"latest_commit_sha":null,"homepage":"https://apchenstu.github.io/FactorFields/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-27T14:47:53.000Z","updated_at":"2025-06-03T07:48:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"3ffd87c1-1576-4d4b-893e-8bcbbdae622b","html_url":"https://github.com/autonomousvision/factor-fields","commit_stats":null,"previous_names":["autonomousvision/factor-fields"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/autonomousvision/factor-fields","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Ffactor-fields","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Ffactor-fields/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Ffactor-fields/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Ffactor-fields/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/factor-fields/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Ffactor-fields/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267428592,"owners_count":24085778,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-27T02:00:11.917Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dictionary","nerf","representation-learning"],"created_at":"2024-11-07T02:05:36.684Z","updated_at":"2025-07-27T21:37:11.699Z","avatar_url":"https://github.com/autonomousvision.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Factor Fields\n## [Project page](https://apchenstu.github.io/FactorFields/) |  [Paper](https://arxiv.org/abs/2302.01226)\nThis repository contains a pytorch implementation for the paper: [Factor Fields: A Unified Framework for Neural Fields and Beyond](https://arxiv.org/abs/2302.01226) and [Dictionary Fields: Learning a Neural Basis Decomposition](https://arxiv.org/abs/2302.01226). Our work present a novel framework for modeling and representing signals, \nwe have also observed that Dictionary Fields offer benefits such as improved **approximation quality**, **compactness**, **faster training speed**, and the ability to **generalize** to unseen images and 3D scenes.\u003cbr\u003e\u003cbr\u003e\n\n\n## Installation\n\n#### Tested on Ubuntu 20.04 + Pytorch 1.13.0 \n\nInstall environment:\n```sh\nconda create -n FactorFields python=3.9\nconda activate FactorFields\nconda install -c \"nvidia/label/cuda-11.7.1\" cuda-toolkit\nconda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.7 -c pytorch -c nvidia\npip install -r requirements.txt \n```\n\nOptionally install [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn), only needed if you want to run hash grid based representations.\n```sh\nconda install -c \"nvidia/label/cuda-11.7.1\" cuda-toolkit\npip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch\n```\n\n\n# Quick Start\nPlease ensure that you download the corresponding dataset and extract its contents into the `data` folder.\n\n## Image\n* [Data - Image Set](https://huggingface.co/apchen/Factor_Fields/blob/main/images.zip)\n\nThe training script can be found at `scripts/2D_regression.ipynb`, and the configuration file is located at `configs/image.yaml`.\n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"media/Girl_with_a_Pearl_Earring.jpg\" alt=\"Girl with a Pearl Earring\" width=\"320\"\u003e\n\u003c/p\u003e\n\n## SDF\n* [Data - Mesh set](https://huggingface.co/apchen/Factor_Fields/blob/main/SDFs.zip)\n\nThe training script can be found at `scripts/sdf_regression.ipynb`, and the configuration file is located at `configs/sdf.yaml`.\n\n\u003cimg src=\"https://github.com/apchenstu/GIFs/blob/main/FactorField-statuette.gif\" alt=\"GIF\" width=\"500px\"\u003e\n\n\n\n## NeRF\n* [Data - Synthetic-NeRF](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1) \n* [Data-Tanks\u0026Temples](https://dl.fbaipublicfiles.com/nsvf/dataset/TanksAndTemple.zip)\n\nThe training script can be found at `train_per_scene.py`:\n\n```python\npython train_per_scene.py configs/nerf.yaml defaults.expname=lego dataset.datadir=./data/nerf_synthetic/lego\n```\n\n\u003cimg src=\"https://github.com/apchenstu/GIFs/blob/main/FactorField-mic.gif\" alt=\"GIF\" width=\"500px\"\n\n\n## Generalization Image\n* [Data - FFHQ](https://github.com/NVlabs/ffhq-dataset)\n\nThe training script can be found at `2D_set_regression.ipynb`\n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"media/inpainting.png\" alt=\"Inpainting\" width=\"640\"\u003e\n\u003c/p\u003e\n\n\n\n## Generalization NeRF\n* [Data - Google Scanned Objects](https://drive.google.com/file/d/1w1Cs0yztH6kE3JIz7mdggvPGCwIKkVi2/view)\n\n```python\npython train_across_scene.py configs/nerf_set.yaml\n```\n\n\u003cimg src=\"https://github.com/apchenstu/GIFs/blob/main/FactorField-few-shot.gif\" alt=\"GIF\" width=\"500px\"\u003e\n\n\n## More examples\n\nCommand explanation with a nerf example:\n* `model.basis_dims=[4, 4, 4, 2, 2, 2]` adjusts the number of levels and channels at each level, with a total of 6 levels and 18 channels.\n* `model.basis_resos=[32, 51, 70, 89, 108, 128]` represents the resolution of the feature embeddings.\n* `model.freq_bands=[2.0, 3.2, 4.4, 5.6, 6.8, 8.0]` indicates the frequency parameters applied at each level of the coordinate transformation function.\n* `model.coeff_type` represents the coefficient field representations and can be one of the following: [none, x, grid, mlp, vec, cp, vm].\n* `model.basis_type` represents the basis field representation and can be one of the following: [none, x, grid, mlp, vec, cp, vm, hash].\n* `model.basis_mapping` represents the coordinate transformation and can be one of the following: [x, triangle, sawtooth, trigonometric]. Please note that if you want to use orthogonal projection, choose the cp or vm basis type, as they automatically utilize the orthogonal projection functions.\n* `model.total_params` controls the total model size. It is important to note that the model's size capability is determined by model.basis_resos and model.basis_dims. The total_params parameter mainly affects the capability of the coefficients.\n* `exportation.render_only` you can rendering item after training by setting this label to 1. Please also specify the `defaults.ckpt` label.\n* `exportation....` you can specify whether to render the items of `[render_test, render_train, render_path, export_mesh]` after training by enable the corressponding label to 1.\n\nSome pre-defined configurations (such as occNet, DVGO, nerf, iNGP, EG3D) can be found in `README_FactorField.py`.\n\n\n## COPY RIGHT\n* [Summer Day](https://www.rijksmuseum.nl/en/collection/SK-A-3005) - Credit goes to Johan Hendrik Weissenbruch and rijksmuseum.\n* [Mars](https://solarsystem.nasa.gov/resources/933/true-colors-of-pluto/) - Credit goes to NASA.\n* [Albert](https://cdn.loc.gov/service/pnp/cph/3b40000/3b46000/3b46000/3b46036v.jpg) - Credit goes to Orren Jack Turner.\n* [Girl With a Pearl Earring](http://profoundism.com/free_licenses.html) - Renovation copyright Koorosh Orooj (CC BY-SA 4.0).\n\n\n## Citation\nIf you find our code or paper helpful, please consider citing both of these papers:\n```\n@article{Chen2023factor,\n  title={Factor Fields: A Unified Framework for Neural Fields and Beyond},\n  author={Chen, Anpei and Xu, Zexiang and Wei, Xinyue and Tang, Siyu and Su, Hao and Geiger, Andreas},\n  journal={arXiv preprint arXiv:2302.01226},\n  year={2023}\n}\n\n@article{Chen2023SIGGRAPH, \n title={{Dictionary Fields: Learning a Neural Basis Decomposition}}, \n author={Anpei, Chen and Zexiang, Xu and Xinyue, Wei and Siyu, Tang and Hao, Su and Andreas, Geiger}, \n booktitle={International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH)}, \n year={2023}}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Ffactor-fields","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Ffactor-fields","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Ffactor-fields/lists"}