{"id":13594026,"url":"https://github.com/city-super/BungeeNeRF","last_synced_at":"2025-04-09T05:32:47.320Z","repository":{"id":47617943,"uuid":"512960084","full_name":"city-super/BungeeNeRF","owner":"city-super","description":"[ECCV22] BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering ","archived":false,"fork":false,"pushed_at":"2022-09-19T07:37:46.000Z","size":1444,"stargazers_count":569,"open_issues_count":9,"forks_count":61,"subscribers_count":11,"default_branch":"main","last_synced_at":"2024-11-06T15:48:22.171Z","etag":null,"topics":["city","multiscale","nerf","neural-rendering"],"latest_commit_sha":null,"homepage":"https://city-super.github.io/citynerf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/city-super.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-12T01:33:15.000Z","updated_at":"2024-11-05T13:15:22.000Z","dependencies_parsed_at":"2022-08-12T13:50:13.368Z","dependency_job_id":null,"html_url":"https://github.com/city-super/BungeeNeRF","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/city-super%2FBungeeNeRF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/city-super%2FBungeeNeRF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/city-super%2FBungeeNeRF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/city-super%2FBungeeNeRF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/city-super","download_url":"https://codeload.github.com/city-super/BungeeNeRF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247987056,"owners_count":21028891,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["city","multiscale","nerf","neural-rendering"],"created_at":"2024-08-01T16:01:27.759Z","updated_at":"2025-04-09T05:32:43.962Z","avatar_url":"https://github.com/city-super.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# BungeeNeRF\nThis repository contains the code release for\n[BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering](https://city-super.github.io/citynerf/img/BungeeNeRF_ECCV22.pdf), aka [CityNeRF](https://city-super.github.io/citynerf/).\n\n![scheme](imgs/training_mechanism.png)\n\n## Abstract\nNeural Radiance Field (NeRF) has achieved outstanding performance in modeling 3D objects and controlled scenes, usually under a single scale. \nIn this work, we focus on multi-scale cases where large changes in imagery are observed at drastically different scales. \nThis scenario vastly exists in real-world 3D environments, such as city scenes, with views ranging from satellite level that captures the overview of a city, \nto ground level imagery showing complex details of an architecture; \nand can also be commonly identified in landscape and delicate minecraft 3D models. \nThe wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail,\nwhich poses great challenges to neural radiance field and biases it towards compromised results. \nTo address these issues, we introduce BungeeNeRF, a progressive neural radiance field that achieves level-of-detail rendering across drastically varied scales. \nStarting from fitting distant views with a shallow base block, as training progresses, new blocks are appended to accommodate the emerging details in the increasingly closer views. \nThe strategy progressively activates high-frequency channels in NeRF’s positional encoding inputs and successively unfold more complex details as the training proceeds.\nWe demonstrate the superiority of BungeeNeRF in modeling diverse multi-scale scenes with drastically varying views on multiple data sources (city models, synthetic, and drone captured data) and its support for high-quality rendering in different levels of detail.\n\n## Installation\nWe recommend using [Anaconda](https://www.anaconda.com/products/individual) to set\nup the environment. Run the following commands:\n\n```\ngit clone https://github.com/city-super/BungeeNeRF.git; cd BungeeNeRF\nconda create --name bungee python=3.7; conda activate bungee\nconda install pip; pip install --upgrade pip\npip install -r requirements.txt\nmkdir data\n```\n\n## Data\nTwo pre-processed data can be download from: [Google Drive](https://drive.google.com/drive/folders/1ybq-BuRH0EEpcp5OZT9xEMi-Px1pdx4D?usp=sharing). Unzip `multiscale_google_56Leonard.zip` and `multiscale_google_Transamerica.zip` to `data` dir. These two folders contain rendered images and processed camera poses. We also offer two .eps files that can be loaded into [Google Earth Studio](https://earth.google.com/studio/). You can adjust camera trajectory and render the most updated views for yourself. The appearance of cities are always updated in GES :). We recommend reading [3D Camera Export](https://earth.google.com/studio/docs/advanced-features/3d-camera-export/) and [FAQs](https://www.google.com/earth/studio/faq/) for camera configuration and permitted usages.\n\n![panel](imgs/panel.png)\n\n\u003cdetails\u003e\n\u003csummary\u003e Exported 3D tracking data (.json) format \u003c/summary\u003e                                                                                    \n{\"name\": xxxx,\n\"width\": xxxx,\n\"height\": xxxx,\n\"numFrames\": xxxx,\n\"durationSeconds\": 56.3,\n\"cameraFrames\": [\n        {\n            \"position\": {\n                \"x\": xxx,\n                \"y\": xxx,\n                \"z\": xxx\n            },\n            \"rotation\": {\n                \"x\": xxx,\n                \"y\": xxx,\n                \"z\": xxx\n            },\n            \"coordinate\": {\n                \"latitude\": xx,\n                \"longitude\": xx,\n                \"altitude\": xxx\n            },\n            \"fovVertical\": xx\n        },\n        ...\n    ],\n\"trackPoints\": []}\n\u003c/details\u003e\n\n\nSome notes on processing camera poses exported from Google Earth Studio:\n* Use the local coordinate when exporting the .json file (i.e. [ECEF coordinate system](http://dirsig.cis.rit.edu/docs/new/coordinates.html)) by setting a track point at the center of the scene. The \"position\" entry and \"rotation\" entry give camera poses.\n* To get the rotation matrix, use the \"rotation\" entry in the exported .json file and consider applying `x'=-x, y'=180-y, z'=180+z` to get Euler angles.\n* For the ease of computing the ray-sphere intersection of different cities, we further transfer the coordinate to [ENU](http://dirsig.cis.rit.edu/docs/new/coordinates.html) coordinate system. Check out [this](https://github.com/geospace-code/pymap3d/blob/743bb002d5f6ea1a7f788bcebd1ff5f62a66460b/src/pymap3d/ecef.py#L206) function. \n* Scale down the whole scene to lie within a period (e.g. [-pi, pi]) to be effectively represented by the positional encoding.\n* H, W, fov can be directly read from the exported .json file and used to compute focal.\n\nFeel free to contact authors if you have any question about the data.\n\n\n## Running\nTo run experiments, use:\n```\npython run_bungee.py --config configs/EXP_CONFIG_FILE\n```\nThe training starts from the furthest scale, with `cur_stage=0`. After an ideal amount of training you can switch to the next training stage by specifying `cur_stage=1`, which will include one finer scale into the training set; and start training from a previous stage checkpoint specified with `--ft_path`:\n```\npython run_bungee.py --config configs/EXP_CONFIG_FILE --cur_stage 1 --ft_path PREV_CKPT_PATH\n```\n\n## Rendering\nTo render views, use:\n```\npython run_bungee.py --config configs/EXP_CONFIG_FILE --render_test\n```\n\n\n## Citation\n```\n@inproceedings{xiangli2022bungeenerf,\n    title={BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering},\n    author={Xiangli, Yuanbo and Xu, Linning and Pan, Xingang, and Zhao, Nanxuan and Rao, Anyi and Theobalt, Christian and Dai, Bo and Lin, Dahua},\n    booktitle = {The European Conference on Computer Vision (ECCV)}, \n    year={2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcity-super%2FBungeeNeRF","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcity-super%2FBungeeNeRF","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcity-super%2FBungeeNeRF/lists"}