{"id":13442628,"url":"https://github.com/ajhamdi/sparf_pytorch","last_synced_at":"2025-04-09T23:51:46.481Z","repository":{"id":64976374,"uuid":"579645101","full_name":"ajhamdi/sparf_pytorch","owner":"ajhamdi","description":"official repo for the paper \"SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images\"","archived":false,"fork":false,"pushed_at":"2024-08-15T12:35:49.000Z","size":10510,"stargazers_count":68,"open_issues_count":0,"forks_count":4,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-04-09T23:51:37.142Z","etag":null,"topics":["3d","deep-learning","nerf","radiance-field"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajhamdi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-18T11:57:35.000Z","updated_at":"2025-01-15T18:45:25.000Z","dependencies_parsed_at":"2024-01-18T14:41:09.027Z","dependency_job_id":"8be6f3c7-d9d0-41cc-9527-943e661135e8","html_url":"https://github.com/ajhamdi/sparf_pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajhamdi%2Fsparf_pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajhamdi%2Fsparf_pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajhamdi%2Fsparf_pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajhamdi%2Fsparf_pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajhamdi","download_url":"https://codeload.github.com/ajhamdi/sparf_pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248131454,"owners_count":21052819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","deep-learning","nerf","radiance-field"],"created_at":"2024-07-31T03:01:48.253Z","updated_at":"2025-04-09T23:51:46.455Z","avatar_url":"https://github.com/ajhamdi.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images (ICCV 2023)\nBy [Abdullah Hamdi](https://abdullahamdi.com/), [Bernard Ghanem](http://www.bernardghanem.com/), [Matthias Nießner](https://niessnerlab.org/members/matthias_niessner/profile.html) \n### [Paper](https://openaccess.thecvf.com/content/ICCV2023W/AI3DCC/html/Hamdi_SPARF_Large-Scale_Learning_of_3D_Sparse_Radiance_Fields_from_Few_ICCVW_2023_paper.html) | [Video](https://youtu.be/VcjypZ0hp4w) | [Website](https://abdullahamdi.com/sparf/) | [Dataset (code: sparf)](https://exrcsdrive.kaust.edu.sa/index.php/s/AzPfy0k45X01ql3) . \u003cbr\u003e\n\u003cp float=\"left\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/26301932/208697062-829496a7-4a25-42cf-8a67-41cc64b0ea66.gif\" align=\"left\" width=\"250\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/26301932/208697090-2bb7ade0-1cce-4ebe-bbd8-c61d4fcfb587.gif\" align=\"center\" width=\"250\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/26301932/208697114-ce5e0a29-4cec-41ec-b995-e6b41495b042.gif\" align=\"center\" width=\"250\"\u003e\n\u003c/p\u003e\n \nThe official Pytroch code of the paper [SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images](https://arxiv.org/abs/2212.09100). SPARF is a large-scale sparse radiance field dataset consisting of ~ 1 million SRFs with multiple voxel resolutions (32, 128, and 512) and 17 million posed images with a resolution of 400 X 400. Furthermore, we propose SuRFNet, a pipline to generate SRFs conditioned on input images, achieving SOTA on ShapeNet novel views synthesis from one or few input images. \n\n# Environment setup\n\nfollow instructions in [INTALL.md](https://github.com/ajhamdi/sparf_pytorch/blob/main/INSTALL.md) to setup the conda environment.\n\n## SPARF Posed Multi-View Image Dataset \nThe dataset is released in the [link (code: sparf)](https://exrcsdrive.kaust.edu.sa/index.php/s/AzPfy0k45X01ql3). Each of SPARF's classes has the same structure of [NeRF-synthetic](https://github.com/sxyu/pixel-nerf) dataset and can be loaded similarly. Download all content in the link and place inside `data/SPARF_images`. Then you can run the [notebook example](https://github.com/ajhamdi/sparf_pytorch/blob/main/examples/mvimage_load.ipynb). \n\n\n## SPARF Radiance Field Dataset\nThe dataset is released in the [link](https://drive.google.com/drive/folders/1Qd_hBrRKR1vlCacOSyK_FN4igkHSbPSM?usp=sharing). Each of SPARF's instances has (beside the posed images above) two directories: `STF` (RGB voxels) and `SRF` (Spherical Harmonics voxels). The full radiance fileds are available under `\u003cCLASS_LABEL\u003e/\u003cINSTANCE_ID\u003e/SRF/vox_\u003cVOXEL-RESOLUTION\u003e/full`, where `\u003cVOXEL-RESOLUTION\u003e` is the resolution (32, 128 or 512). The partial SRFs are stored in `\u003cCLASS_LABEL\u003e/\u003cINSTANCE_ID\u003e/STF/vox_\u003cVOXEL-RESOLUTION\u003e/partial` similarly. The partitioning (shards) and splits of the dataset is available on the file `SNRL_splits.csv` in the root of the dataset.  The voxles information are stored as sparse voxels in `data_0.npz`as coords and values. \n\nDownload all content in the link and place inside `data/SPARF_srf`. Then you can run the [main training code](https://github.com/ajhamdi/sparf_pytorch/blob/main/run_sparf.py).\n\n## Script for rendering ShapeNet images used in creating SPARF \nmake sure that `ShapeNetCore.v2` is downloaded and placed in `data/ShapeNetCore.v2`. Then run the following script to render the images used in creating SPARF. \n```bash\npython run_sparf.py --run render --data_dir data/SPARF_srf/ --nb_views 400 --object_class car \n```\n## Script for extracting SPARF Radiance Fields (full SRFs with voxel res=128 and SH dim=4)\nmake sure that `SPARF_images` is downloaded and placed in `data/SPARF_images`. Then run the following script to extract the SRFs.\n```bash\npython run_sparf.py --run extract --vox_res 128 --sh_dim 4 --object_class airplane --data_dir data/SPARF_images/ --visualize --evaluate \n```\n\n## Script for extracting SPARF Radiance Fields (partial SRFs with voxel res=512 and SH dim=1, nb_views=3)\nmake sure that `SPARF_images` is downloaded and placed in `data/SPARF_images`. Then run the following script to extract the SRFs.\n```bash\npython run_sparf.py --run preprocess --vox_res 512 --sh_dim 1 --rf_variant 0 --object_class airplane --nb_views 3 --data_dir data/SPARF_images/ --randomized_views\n```\n## Training and Inference pipeline on SPARF Radiance Fields\nmake sure that `SPARF_srf` is downloaded and placed in `data/SPARF_srf`. Then run the following script to train on SRFs.\n```bash\npython run_sparf.py --vox_res 128 --nb_views 3 --nb_rf_variants 4 --input_quantization_size 1.0 --strides 2 --lr_decay 0.99 --batch_size 6 --lr 1e-2 --visualize --normalize_input const --lambda_cls 30.0 --lambda_main 2.0 --augment_type none --mask_type densepoints --ignore_loss_mask --nb_frames 200 --validate_training  --data_dir data/SPARF_srf/ --run train --object_class airplane \n```\n\n## Citation\nIf you find our work useful in your research, please consider citing:\n```bibtex\n@InProceedings{Hamdi_2023_ICCV,\n    author    = {Hamdi, Abdullah and Ghanem, Bernard and Nie{\\ss}sner, Matthias},\n    title     = {SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images},\n    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},\n    month     = {October},\n    year      = {2023},\n    pages     = {2930-2940}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajhamdi%2Fsparf_pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajhamdi%2Fsparf_pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajhamdi%2Fsparf_pytorch/lists"}