{"id":13760079,"url":"https://github.com/sxyu/volrend","last_synced_at":"2025-05-15T12:03:01.495Z","repository":{"id":40609292,"uuid":"332336649","full_name":"sxyu/volrend","owner":"sxyu","description":"PlenOctree Volume Rendering (supports CUDA \u0026 fragment shader backends)","archived":false,"fork":false,"pushed_at":"2024-10-22T05:34:14.000Z","size":7806,"stargazers_count":613,"open_issues_count":17,"forks_count":84,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-04-14T19:59:29.610Z","etag":null,"topics":["octree","rendering","spherical-harmonics","volume-rendering"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sxyu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-24T00:28:12.000Z","updated_at":"2025-03-23T11:36:28.000Z","dependencies_parsed_at":"2023-01-19T04:45:49.714Z","dependency_job_id":"9a75fa17-3cef-4dbc-93c6-692f20850c39","html_url":"https://github.com/sxyu/volrend","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxyu%2Fvolrend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxyu%2Fvolrend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxyu%2Fvolrend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxyu%2Fvolrend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sxyu","download_url":"https://codeload.github.com/sxyu/volrend/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["octree","rendering","spherical-harmonics","volume-rendering"],"created_at":"2024-08-03T13:01:03.100Z","updated_at":"2025-05-15T12:03:01.474Z","avatar_url":"https://github.com/sxyu.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"# PlenOctree Volume Rendering\n\nThis is a real-time PlenOctree volume renderer written in C++ using OpenGL,\nconstituting part of the code release for:\n\nPlenOctrees for Real Time Rendering of Neural Radiance Fields\u003cbr\u003e\nAlex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, Angjoo Kanazawa\n\nhttps://alexyu.net/plenoctrees\n\n```\n@inproceedings{yu2021plenoctrees,\n      title={{PlenOctrees} for Real-time Rendering of Neural Radiance Fields},\n      author={Alex Yu and Ruilong Li and Matthew Tancik and Hao Li and Ren Ng and Angjoo Kanazawa},\n      year={2021},\n      booktitle={ICCV},\n}\n```\n\n![Screenshot](https://raw.githubusercontent.com/sxyu/volrend/master/img/screenshot_slice.jpg)\n\nThe project has several repositories:\n\n- NeRF-SH training and PlenOctree extraction \u003chttps://github.com/sxyu/plenoctree\u003e\n- PyTorch PlenOctree rendering CUDA extension \u003chttps://github.com/sxyu/svox\u003e\n- General NeRF visualization library \u003chttps://github.com/sxyu/nerfvis\u003e\n\nMore will be released soon, we are taking a short break now.\n\n## Building\nPlease install a recent version of CMake \u003chttps://cmake.org\u003e\n\n### Linux\n```sh\nmkdir build \u0026\u0026 cd build\ncmake ..\nmake -j12\n```\n\n- If you do not have CUDA-capable GPU, pass `-DVOLREND_USE_CUDA=OFF` after `cmake ..` to use fragment shader backend, which is also used for the web demo.\n  It is slower and does not support mesh-insertion and dependent features such as lumisphere probe.\n\nThe main real-time PlenOctree rendererer `volrend` and a headless version `volrend_headless` are built. The latter requires CUDA.\nThere is also an animation maker `volrend_anim`, which I used to make some of the video animations; don't worry about it unless interested.\n\nYou should be able to build the project as long as you have GLFW.\nOn Ubuntu, you will need X-server; you can try\n`sudo apt-get install libgl1-mesa-dev libxi-dev libxinerama-dev libxcursor-dev libxrandr-dev libgl1-mesa-dev libglu1-mesa-dev`\n\n### macOS\nFor macOS, we assume you have the homebrew package manager, and no CUDA-capable GPU.\n```sh\nbrew install cmake\nbrew install glfw\nmkdir build \u0026\u0026 cd build\ncmake .. -DVOLREND_USE_CUDA=OFF\nexport LIBRARY_PATH=$LIBRARY_PATH:/usr/local/lib; export CPLUS_INCLUDE_PATH=\"/usr/local/Cellar/glfw/3.3.4/include\"; make -j8\n```\n\n### Windows 10\nInstall Visual Studio (I am using 2019 here). Then\n```sh\nmkdir build \u0026\u0026 cd build\ncmake .. -G\"Visual Studio 16 2019\"\ncmake --build . --config Release\n```\n- If you do not have CUDA-capable GPU, pass `-DVOLREND_USE_CUDA=OFF` after `cmake ..` to use fragment shader backend, which is also used for the web demo.\n  It is slower and does not support mesh-insertion and dependent features such as lumisphere probe.\n\nThe main real-time PlenOctree rendererer `volrend` and a headless version `volrend_headless` are built. The latter requires CUDA.\nThere is also an animation maker `volrend_anim`, which I used to make some of the video animations; don't worry about it unless interested.\n\n### Dependencies\n- C++17\n- OpenGL\n    - any dependencies of GLFW\n- libpng-dev (only for writing image in headless mode and saving screenshot)\n\n#### Optional\n- CUDA Toolkit, I tried on both 11.0 and 10.2\n    - Pass `-DVOLREND_USE_CUDA=OFF` to disable it.\n\n## Run\n```sh\n./volrend \u003cname\u003e.npz\n```\nSee `--help` for flags.\n\nThere is an ImGui window which exposes rendering options as well as interactive features mentioned in the paper + video.\nFor the mesh insertion, only OBJ files optionally with vertex coloring are supported. Texturing mapping is not implemented right now.\nSome example meshes are in `sample_obj`, and a program to generate SH meshes (just for fun) is in `sample_obj/sh/gen_sh.cpp`.\nPlease use meshlab to triangulate other mesh.\n\n### Keyboard + Mouse Controls (Desktop GUI)\n- Left mouse btn + drag: rotate about camera position\n- Right mouse btn + drag: rotate about origin point (can be moved)\n- Middle mouse btn + drag: pan camera\n- Shift + Left mouse btn + drag: pan camera (alt)\n- Shift + middle mouse btn + drag: pan camera AND move origin point simultaneously\n- Scroll with wheel: move forward/back in z\n- WASDQE: move; Shift + WASDQE to move faster\n- 123456: preset `world_up` directions, sweep through these keys if scene is using different coordinate system.\n- 0: reset the focal length to default, if you messed with it\n- Z: cycle gizmo operations translate/rotate/scale (only visible if mesh is opened in Manipulate section)\n- X: toggle gizmo space local/world (applies to mesh translation/rotation)\n\nLumisphere probe:\n- IJKLUO: move the lumisphere probe; Hold shift to move faster\n\n\n### Offscreen Rendering\n\nThe program `volrend_headless` allows you to perform offscreen rendering on a server.\n\nUsage: `./volrend_headless tree.npz -i intrinsics.txt pose1 pose2... [-o out_dir]`\n\nintrinsics.txt should be a 4x4 intrinsics matrix.\npose1, pose2 ... should contain 3x4 or 4x4 c2w pose matrices,\nor multiple matrices in a 4Nx4 format.\nAdd `-r` to use OpenCV camera space instead of NeRF.\n\nThe following zip file contains intrinsics and pose files for each scene of NeRF-synthetic,\n\u003chttps://drive.google.com/file/d/1mI4xl9FXQDm_0TidISkKCp9eyTz40stE/view?usp=sharing\u003e\n\nExample to render out images:\n`./volrend_headless drums/tree.npz -i data/nerf_synthetic/drums/intrinsics.txt data/nerf_synthetic/drums/pose/* -o tree_rend/drums`\n\nThe PNG writing is a huge bottleneck. Example to compute the FPS:\n`./volrend_headless drums/tree.npz -i data/nerf_synthetic/drums/intrinsics.txt data/nerf_synthetic/drums/pose/*`\n\nSee `./volrend_headless --help` for more options such as setting rendering options.\n\n## Precomputed PlenOctree Files\nThe full resolution tree files for NeRF-synthetic reported in the paper may be found at:\n\u003chttps://drive.google.com/drive/folders/1DIYj-iu3TOHProJVHPIQTjHnmYf80_vC?usp=sharing\u003e\n\nThe uncompressed NeRF-synthetic files used for the web demo are here:\n\u003chttps://drive.google.com/drive/folders/1vGXEjb3yhbClrZH1vLdl2iKtowfinWOg?usp=sharing\u003e\nThe compression script used to turn this in to the web version is in `scripts/compress_octree.py`.\n\nMore to come soon.\n\n## PyTorch Extension: svox\n\nYou can find a (mostly) compatible PlenOctree library called `svox`, which we use to build the tree;\n`pip install svox`.\n\n- Code: \u003chttps://github.com/sxyu/svox\u003e\n- Documentation: \u003chttps://svox.readthedocs.io\u003e\n\nMore information to be added soon.\n\n## Building the Web Demo\n\nThe backend of the web demo is built from the shader version of the C++ source using emscripten.\nInstall emscripten per instructions here:\nhttps://emscripten.org/docs/getting_started/downloads.html\n\nThen use\n```sh\nmkdir embuild \u0026\u0026 cd embuild\nemcmake cmake ..\nmake -j12\n```\n\nThe full website should be written to `embuild/build`.\nSome CMake scripts even write the html/css/js files.\nTo launch it locally for previewing, you can use the make target:\n```sh\nmake serve\n```\nWhich should launch a server at http://0.0.0.0:8000/.\nNEW: You may load local files and OBJs by using `Load Local` in the navbar.\nOpen `Layers` in the top right and click the camera layer to show cameras.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxyu%2Fvolrend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsxyu%2Fvolrend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxyu%2Fvolrend/lists"}