{"id":25283757,"url":"https://github.com/half-potato/ever_training","last_synced_at":"2025-10-27T18:31:46.666Z","repository":{"id":276881385,"uuid":"925473746","full_name":"half-potato/ever_training","owner":"half-potato","description":"Original reference implementation of \"EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis\"","archived":false,"fork":false,"pushed_at":"2025-02-10T23:29:54.000Z","size":2935,"stargazers_count":18,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-11T00:27:35.074Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/half-potato.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-01T00:29:36.000Z","updated_at":"2025-02-10T23:29:57.000Z","dependencies_parsed_at":"2025-02-11T00:29:44.946Z","dependency_job_id":"07527e7b-ea53-4bde-a371-2a9df5acf88e","html_url":"https://github.com/half-potato/ever_training","commit_stats":null,"previous_names":["half-potato/ever_training"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/half-potato%2Fever_training","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/half-potato%2Fever_training/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/half-potato%2Fever_training/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/half-potato%2Fever_training/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/half-potato","download_url":"https://codeload.github.com/half-potato/ever_training/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238542280,"owners_count":19489557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-12T20:01:58.349Z","updated_at":"2025-10-27T18:31:46.660Z","avatar_url":"https://github.com/half-potato.png","language":"Jupyter Notebook","funding_links":[],"categories":["3DGSNeRF"],"sub_categories":[],"readme":"# Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis\nThis is the repository with changes to 3DGS's training code to use the EVER rendering method.\n\nEver is a method for real-time differentiable emission-only volume rendering. Unlike recent\nrasterization based approach by 3D Gaussian Splatting (3DGS), our primitive based representation\nallows for exact volume rendering, rather than alpha compositing 3D Gaussian billboards. As such,\nunlike 3DGS our formulation does not suffer from popping artifacts and view dependent density, but\nstill achieves frame rates of ∼30 FPS at 720p on an NVIDIA RTX4090. Because our approach is built\nupon ray tracing it supports rendering techniques such as defocus blur and camera distortion (e.g.\nsuch as from fisheye cameras), which are difficult to achieve by rasterization. We show that our\nmethod has higher performance and fewer blending issues than 3DGS and other subsequent works,\nespecially on the challenging large-scale scenes from the Zip-NeRF dataset where it achieves SOTA\nresults among real-time techniques.\n\nDatasets:\n[mipnerf360pt1](http://storage.googleapis.com/gresearch/refraw360/360_v2.zip), \n[mipnerf360pt2](https://storage.googleapis.com/gresearch/refraw360/360_extra_scenes.zip), \n[zipnerf](https://smerf-3d.github.io/)\n\n`zipnerf-undistorted` is used for evaluation against 3DGS.\n\nMore details can be found in our [paper](https://arxiv.org/abs/2410.01804) or at our [website](https://half-potato.gitlab.io/posts/ever/)\n\n\u003csection class=\"section\" id=\"BibTeX\"\u003e\n  \u003cdiv class=\"container is-max-desktop content\"\u003e\n    \u003ch2 class=\"title\"\u003eBibTeX\u003c/h2\u003e\n    \u003cpre\u003e\u003ccode\u003e\n@misc{mai2024everexactvolumetricellipsoid, title={EVER: Exact Volumetric Ellipsoid Rendering for Real-time View Synthesis},  author={Alexander Mai and Peter Hedman and George Kopanas and Dor Verbin and David Futschik and Qiangeng Xu and Falko Kuester and Jon Barron and Yinda Zhang}, year={2024}, eprint={2410.01804}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2410.01804},  }\n\u003c/code\u003e\u003c/pre\u003e\n  \u003c/div\u003e\n\u003c/section\u003e\n\n\n## Quick Install\n\nIf you wish to skip the install, use this command to train:\n\n```\ngit clone --recursive https://github.com/half-potato/ever_training\npython docker_train.py -s \u003cpath to dataset\u003e -m \u003cpath to output\u003e ...\n```\nThe docker train script should have the same arguments as `train.py`. To visualize, do:\n```\nbash docker_visualize.py \u003cpath to output\u003e \u003cpath to dataset\u003e 6006 127.0.0.1\n```\nand in a different terminal:\n```\ncd SIBR_viewers\n./install/bin/SIBR_remoteGaussian_app --ip 127.0.0.1 --port 6009\n```\n\n\n### Dependencies\n- OptiX 7.4, which must be downloaded from NVIDIA's [website](https://developer.nvidia.com/designworks/optix/downloads/legacy). This is downloaded and placed somewhere on your computer, then use `export OptiX_INSTALL_DIR=...` to set the variable to that location.\n- [*SlangD*](https://github.com/shader-slang/slang). We recommend using this [version](https://github.com/shader-slang/slang/releases/tag/v2025.6)\nWe can install the rest of the dependencies as follows:\n```\nsudo apt install -y libglew-dev libassimp-dev libboost-all-dev libgtk-3-dev libopencv-dev libglfw3-dev libavdevice-dev libavcodec-dev libeigen3-dev libxxf86vm-dev libembree-dev libglm-dev\nconda env create --name ever python==3.10\nconda activate ever\nconda install pip\n# adjust for cuda version\npip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126\n```\n\u003c!-- conda env create --file environment.yml --\u003e\n\nFor Manjaro, Arch, or other rolling release, run the following:\n```\nexport CXX=/usr/bin/g++-11 CC=/usr/bin/gcc-11 \n```\n\nIf you have multiple cuda versions, make sure to specify which one to use using the following command:\n```\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-12.6/lib64\nexport PATH=$PATH:/usr/local/cuda-12.6/bin\n```\n\nNow, download the files and run `bash install.bash`\n```\ngit clone --recursive https://github.com/half-potato/ever_training\ncd ever_training\npip install -r requirements.txt\nbash install.bash\n```\nIf you get a bunch of compilation errors, it could be that you need to run the export line for the CXX and CC versions.\n\nWe can now train using the following command:\n```\npython train.py -s \u003cpath to COLMAP or NeRF Synthetic dataset\u003e\n```\n\nTested on Manjaro and Ubuntu Linux 22.04.\n\n### Notes to Users\n- Code will silently fail and create NaNs if out of memory.\n- slang can cause hangs on startup after changing code. If this happens, kill the program, delete all '.slangtorch' and '.lock' files and retry.\n\n### Evaluation\nBy default, the trained models use all available images in the dataset. To train them while withholding a test set for evaluation, use the ```--eval``` flag. This way, you can render training/test sets and produce error metrics as follows:\n```shell\npython train.py -s \u003cpath to COLMAP or NeRF Synthetic dataset\u003e --eval # Train with train/test split --images (images_4 for 360 outdoor, images_2 for 360 indoor)\npython render.py -m \u003cpath to trained model\u003e # Generate renderings\npython metrics.py -m \u003cpath to trained model\u003e # Compute error metrics on renderings\n```\nTo run training on a dataset with images with varying exposure levels, we must first extract the metadata from the images. This can be done with this command:\n```\npython extract_metadata.py $DATASET/images $DATASET/metadata.json\n```\n\nIf you want to evaluate our [pre-trained models](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/pretrained/models.zip), you will have to download the corresponding source data sets and indicate their location to ```render.py``` with an additional ```--source_path/-s``` flag. Note: The pre-trained models were created with the release codebase. This code base has been cleaned up and includes bugfixes, hence the metrics you get from evaluating them will differ from those in the paper.\n```shell\npython render.py -m \u003cpath to pre-trained model\u003e -s \u003cpath to COLMAP dataset\u003e\npython metrics.py -m \u003cpath to pre-trained model\u003e\n```\nThese have the same arguments as 3DGS.\n\nTo run the full benchmark, use the following command:\n```shell\npython full_eval.py -m360 $NERF_DATSASETS/360 -zn $NERF_DATSASETS/zipnerf_ud -db $NERF_DATSASETS/db -tnt $NERF_DATSASETS/tandt --output_path eval\n```\n\n## Interactive Viewers\nFor all viewing purposes, we rely on the [SIBR](https://sibr.gitlabpages.inria.fr/) remote viewer. The training script will host a server to view training, and we provide the `host_render_server.py` file for viewing trained models.\n\nThe viewer can then be run as follows:\n```\npython host_render_server.py -m $TRAINED_MODEL_LOCATION -s $SCENE_LOCATION --port $PORT --ip $IP`\n```\nThe `$SCENE_LOCATION` only needs to be provided if viewing on a different machine than the model was trained on. `$IP` is for viewing on remote machines. By default, it is `127.0.0.1`. By default, `$PORT` is 6009. Once the render server has been hosted, we can then run the SIBR remote viewer in a separate terminal and connect it.\nThen, on a different terminal, run:\n```\n./install/bin/SIBR_remoteGaussian_app --ip $IP --port $PORT\n```\n\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cspan style=\"font-weight: bold;\"\u003ePrimary Command Line Arguments for Network Viewer\u003c/span\u003e\u003c/summary\u003e\n\n  #### --path / -s\n  Argument to override model's path to source dataset.\n  #### --ip\n  IP to use for connection to a running training script.\n  #### --port\n  Port to use for connection to a running training script. \n  #### --rendering-size \n  Takes two space separated numbers to define the resolution at which network rendering occurs, ```1200``` width by default.\n  Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too.\n  #### --load_images\n  Flag to load source dataset images to be displayed in the top view for each camera.\n\u003c/details\u003e\n\u003cbr\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhalf-potato%2Fever_training","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhalf-potato%2Fever_training","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhalf-potato%2Fever_training/lists"}