{"id":13442540,"url":"https://github.com/JOP-Lee/READ","last_synced_at":"2025-03-20T14:31:23.423Z","repository":{"id":40496227,"uuid":"490574117","full_name":"JOP-Lee/READ","owner":"JOP-Lee","description":"AAAI2023，implementation of \"READ:  Large-Scale Neural Scene Rendering for Autonomous Driving\", the experimental results are significantly better than Nerf-based methods","archived":false,"fork":false,"pushed_at":"2023-04-18T14:58:49.000Z","size":84544,"stargazers_count":451,"open_issues_count":28,"forks_count":56,"subscribers_count":21,"default_branch":"main","last_synced_at":"2024-10-28T05:59:20.956Z","etag":null,"topics":["driving","kitti","nerf","neural-rendering","view-synthesis"],"latest_commit_sha":null,"homepage":"https://github.com/JOP-Lee/READ-Large-Scale-Neural-Scene-Rendering-for-Autonomous-Driving","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JOP-Lee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-05-10T06:44:58.000Z","updated_at":"2024-10-23T14:22:45.000Z","dependencies_parsed_at":"2024-01-18T14:41:12.512Z","dependency_job_id":"62e11095-7955-4ff2-9fc1-b9e86405727f","html_url":"https://github.com/JOP-Lee/READ","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JOP-Lee%2FREAD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JOP-Lee%2FREAD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JOP-Lee%2FREAD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JOP-Lee%2FREAD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JOP-Lee","download_url":"https://codeload.github.com/JOP-Lee/READ/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244630159,"owners_count":20484326,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["driving","kitti","nerf","neural-rendering","view-synthesis"],"created_at":"2024-07-31T03:01:47.023Z","updated_at":"2025-03-20T14:31:21.883Z","avatar_url":"https://github.com/JOP-Lee.png","language":"Python","funding_links":[],"categories":["Python","Softwares and Libraries"],"sub_categories":[],"readme":"# READ: Large-Scale Neural Scene Rendering for Autonomous Driving\n\n\n\nCommercial use is forbidden![](https://img.shields.io/badge/license-GPL-blue)\n\nPaper: https://arxiv.org/abs/2205.05509 (Old version)\n\nVideo: \n [Bilibili](https://www.bilibili.com/video/BV1we411K7ug)  [Bilibili2](https://www.bilibili.com/video/BV1C84y1t7pP)  [Youtube](https://www.youtube.com/watch?v=2yeH7l4oLYw)  \n \u003c!--- \n [(Compressed)](https://youtu.be/73zcrqwNFRk)\n--\u003e\n\n\nDemo: (Use only one camera view for training)\n\nAll you need is a series of images or videos as input to get the following effect. Scenes are not only limited to driving scenes, but also can be used for tourism scenes, indoor scenes, objects, etc.\n\n\u003cp float=\"left\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/24960306/168012420-468478de-1db5-430d-bdd2-b52755477cd3.gif\" width=\"270\"/\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/24960306/168014170-b964a639-25de-4290-8e91-dc3d3f66ab7c.gif\" width=\"270\"/\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/24960306/168012387-ff471fcf-f617-4844-a4d6-bfbf52753d03.gif\" width=\"270\"/\u003e\n\u003c/p\u003e\n\n\u003cp float=\"center\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24960306/168002213-c7c49209-d2bf-489d-9f84-aac2fe6b757b.gif\" width=\"410\"\u003e\n   \u003cimg src=\"https://user-images.githubusercontent.com/24960306/169205523-2d4e051a-2c56-461d-b16a-bb022e2596f2.gif\" width=\"410\"\u003e\n\u003c/p\u003e\n\n\n\n\n\n## Overview: \n\nThis is the code release for our AAAI2023 paper, PyTorch implementation of Large-Scale Neural Scene Rendering for Autonomous Driving(READ), a large-scale neural rendering method is proposed to synthesize the autonomous driving scene~(READ), which makes it possible to synthesize large-scale driving scenarios on a PC. Our model can not only synthesize realistic driving scenes but also stitch and edit driving scenes.\n![contents](./image/main.jpg)\n\n## Setup\n\nThe following instructions describe installation of conda environment.  Please refer to [requirement](https://github.com/JOP-Lee/READ/blob/main/requirement.sh).\n\nIf you want to set it to headless mode(without X server enabled), see the MyRender in the [src folder](https://github.com/JOP-Lee/READ/tree/main/src). \n\nNote: This project needs to use the screen to run the script, if not, run the project in [src folder](https://github.com/JOP-Lee/READ/tree/main/src).\n\nRun this command to install python environment:\n```bash\ncd src/MyRender\npip install -v -e .  \n```\n\n \n## Run\n\nYou can render one of the fitted scenes we provide right away in the real-time viewer or fit your own scene.\n\nDownload fitted scenes and universal rendering network weights from [here](https://zenodo.org/record/7395608#.Y4xv9HZBxPY) and unpack in the Data directory.\n\nWe suppose that you have at least one GeForce GTX 1080 Ti for fitting and inference.\n\n\n\n### Use fitted scene\n\nHere we show an example how to run fitted scenes in the viewer.\n\n#### kitti6\n```bash\npython viewer.py --config downloads/kitti6.yaml\n```\n\n### Viewer navigation:\n\n* Move forward: w \n* Move backward: s\n* Move left: a \n* Move right: d\n* Rise: q\n* Fall: e\n* Turn to the left: 1\n* Turn to the right: 2\n* Turn to the up: 3\n* Turn to the down: 4\n* Rotation: press left mouse button and drag\n* Move: press rigth mouse button and drug / scroll middle mouse botton\n* Pan: press middle mouse button and drug\n\n## Train\n\npython train.py --config configs/train_example.yaml --pipeline READ.pipelines.ogl.TexturePipeline --crop_size 256x256\n\nThe size of crop_size depends on your GPU memory, and the parameter train_dataset_args can be adjusted in the configs folder.\n\n\n## Train with your own data\n\nIt's very simple. You just need a sequence of pictures to do it.\n\n1. Use metashape to obtain camera.xml, pointcloud.ply\n2. Place the above files and photos in the Data folder, for example, Data/image/xx.png, Data/camera.xml Data/pointcloud.ply\n3. Change the folder address in configs/paths_example.yaml and load net_ckpt/texture_ckpt model address in configs/train_example.yaml, if any.\n4. Run the code: python train.py --config configs/train_example.yaml --pipeline READ.pipelines.ogl.TexturePipeline --crop_size 256x256\n\n\n\u003c!--- \n# [![Watch the video](https://i.ytimg.com/an_webp/kC-bwky4e7Q/mqdefault_6s.webp?du=3000\u0026sqp=CIDh7JMG\u0026rs=AOn4CLAE5KzsOlrQzpZVB2DYJbC4UMOhGQ)](https://youtu.be/kC-bwky4e7Q)\n[\u003cimg src=\"https://i.ytimg.com/an_webp/kC-bwky4e7Q/mqdefault_6s.webp?du=3000\u0026sqp=CIDh7JMG\u0026rs=AOn4CLAE5KzsOlrQzpZVB2DYJbC4UMOhGQ\" width=\"60%\"\u003e](https://youtu.be/73zcrqwNFRk)\n\n\n\n\n## Novel View(Click to view the video):\n\n[![Watch the video](./image/Video.png)](https://youtu.be/W3h5nmmM5BM )\n--\u003e \n\n\n##  Scene Editing:\n\nREAD can move and remove the cars in different views. A panorama with larger view can be synthesized by changing the camera parameters.\n![contents](./image/Scene_Editing.jpg)\n \n\n## Scene Stitching:\n\nREAD is able to synthesize the larger driving scenes and update local areas with obvious changes in road conditions. \n![contents](./image/Scene_Stitching.jpg)\n\n## Novel View Synthesis:\n\n![contents](./image/NovelView.jpg)\n\n\n## Acknowledgments\nIn this code we refer to the following implementations: [npbg](https://github.com/alievk/npbg) and [\nMIMO](https://github.com/chosj95/MIMO-UNet). Great thanks to them! \n\n\n## Citation\nIf our work or code helps you, please consider to cite our paper. Thank you!\n```BibTeX\n@inproceedings{li2022read,\n  author = {Li, Zhuopeng and Li, Lu and Zhu, Jianke*},\n  title = {READ: Large-Scale Neural Scene Rendering for Autonomous Driving},\n  booktitle = {AAAI},\n  year = {2023}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJOP-Lee%2FREAD","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJOP-Lee%2FREAD","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJOP-Lee%2FREAD/lists"}