{"id":18600798,"url":"https://github.com/autonomousvision/lara","last_synced_at":"2025-04-06T19:12:27.874Z","repository":{"id":247637633,"uuid":"825598033","full_name":"autonomousvision/LaRa","owner":"autonomousvision","description":"[ECCV 2024] Efficient Large-Baseline Radiance Fields,  a feed-forward 2DGS model","archived":false,"fork":false,"pushed_at":"2024-07-13T10:04:50.000Z","size":27696,"stargazers_count":307,"open_issues_count":10,"forks_count":13,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-30T17:11:27.153Z","etag":null,"topics":["3d-reconstruction","radiance-field","transformer"],"latest_commit_sha":null,"homepage":"https://apchenstu.github.io/LaRa/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-08T06:43:52.000Z","updated_at":"2025-03-28T16:00:28.000Z","dependencies_parsed_at":"2024-07-13T10:58:04.213Z","dependency_job_id":null,"html_url":"https://github.com/autonomousvision/LaRa","commit_stats":{"total_commits":7,"total_committers":2,"mean_commits":3.5,"dds":0.1428571428571429,"last_synced_commit":"c32ae7704faee3423634ba226eefc4dda644f8f5"},"previous_names":["autonomousvision/lara"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2FLaRa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2FLaRa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2FLaRa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2FLaRa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/LaRa/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247535519,"owners_count":20954576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","radiance-field","transformer"],"created_at":"2024-11-07T02:05:37.802Z","updated_at":"2025-04-06T19:12:27.844Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LaRa: Efficient Large-Baseline Radiance Fields\n\n[Project page](https://apchenstu.github.io/LaRa/) | [Paper](https://arxiv.org/abs/2407.04699) | [Data](https://huggingface.co/apchen/LaRa/tree/main/dataset) | [Checkpoint](https://huggingface.co/apchen/LaRa/tree/main/ckpts) |\u003cbr\u003e\n\n![Teaser image](assets/demo.gif)\n\n## ⭐ New Features \n- 2024/04/05: Important updates - \nNow our method supports half precision training, achieving over **100% faster** convergence and about **1.5dB** gains with less iterations!\n\n    | Model    | PSNR ↑     | SSIM ↑    | Abs err (Geo) ↓   | Epoch  | Time（day）      | ckpt |\n    | ------   | ------     | ------    | ------    | ------ | ------ | ------ |\n    | Paper    | 27.65      |  0.951    | 0.0654    |  50    |   3.5  | ------ |\n    | bf16     | 29.15      |  0.956    | 0.0574    |  30    |   1.5  | [Download](https://huggingface.co/apchen/LaRa/tree/main/ckpts/) |\n\n    Please download the pre-trained checkpoint from the provided link and place it in the `ckpts` folder.\n\n# Installation\n\n```\ngit clone https://github.com/autonomousvision/LaRa.git --recursive\nconda env create --file environment.yml\nconda activate lara\n```\n\n\n# Dataset \nWe used the processed [gobjaverse dataset](https://aigc3d.github.io/gobjaverse/) for training. A download script `tools/download_dataset.py` is provided to automatically download the datasets.\n\n```\npython tools/download_dataset.py all\n```\nNote: The GObjaverse dataset requires approximately 1.4 TB of storage. You can also download a subset of the dataset. Please refer to the provided script for details. Please manually delete the `_temp` folder after completing the download.\n\nIf you would like to process the data by yourself, we provide preprocess scripts for the gobjaverse and co3d datasets, please check `tools/prepare_dataset_*`.\nYou can also download our preprocessed data and put them to `dataset` folder:\n* [gobjaverse](#gobjaverse)\n* [Google Scaned Object](#GSO)\n* [Co3D](#Co3D) \n* Instant3D - Please contact the authors of Instant3D if you wish to obtain the data for comparison.\n# Training\n```\npython train_lightning.py\n```\n**note:** You can configure the GPU id and other parameter with `configs/base.yaml`.\n\n# Evaluation\nOur method supports the reconstruction of radiance fields from **multi-view**, **text**, and **single view** inputs. We provide a pre-trained checkpoint at [ckpt](https://huggingface.co/apchen/LaRa/resolve/main/ckpts/epoch%3D29.ckpt).\n\n## multi-view to 3D\nTo reproduce the table results, you can simply use:\n```\npython eval_all.py\n```\n**note:** \n- Please double-check that the paths inside the script are correct for your specific case.\n- Please specify the video_frames and save_mesh [labels](https://github.com/autonomousvision/LaRa/blob/main/eval_all.py#L11) if you would like to output mesh or video during the evaluation\n\n## text to 3D\n```\npython evaluation.py configs/infer.yaml \n       infer.ckpt_path=ckpts/epoch=29.ckpt\n       infer.save_folder=outputs/prompts/\n       infer.dataset.generator_type=xxx\n       infer.dataset.prompts=[\"a car made out of sushi\",\"a beautiful rainbow fish\"]\n```\n**note:** This part is currently unavailable due to a permissions issue. I will look for an alternative text-to-multi-view generator later next week.\n\n\n## single view to 3D\n```\npython evaluation.py configs/infer.yaml \n       infer.ckpt_path=ckpts/epoch=29.ckpt\n       infer.save_folder=outputs/single-view/\n       infer.dataset.generator_type=\"zero123plus-v1\"\n       infer.dataset.image_pathes=\\[\"assets/examples/13_realfusion_cherry_1.png\"\\]\n```\n**note:** It supports the generator types `zero123plus-v1.1` and `zero123plus-v1`.\n\n\n\n## Acknowledgements\nOur render is built upon [2DGS](https://github.com/hbb1/2d-gaussian-splatting). The data preprocessing code for the Co3D dataset is partially borrowed from [Splatter-Image](https://github.com/szymanowiczs/splatter-image/blob/main/data_preprocessing/preprocess_co3d.py). Additionally, the script for generating multi-view images from text and single view image is sourced from [GRM](https://github.com/justimyhxu/grm). We thank all the authors for their great repos. \n\n## Citation\nIf you find our code or paper helps, please consider citing:\n```bibtex\n@inproceedings{LaRa,\n         author = {Anpei Chen and Haofei Xu and Stefano Esposito and Siyu Tang and Andreas Geiger},\n         title = {LaRa: Efficient Large-Baseline Radiance Fields},\n         booktitle = {European Conference on Computer Vision (ECCV)},\n         year = {2024}\n        } \n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Flara","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautonomousvision%2Flara","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautonomousvision%2Flara/lists"}