{"id":15028360,"url":"https://github.com/thu-lyj-lab/t3bench","last_synced_at":"2025-05-16T18:10:11.573Z","repository":{"id":198289523,"uuid":"700378752","full_name":"THU-LYJ-Lab/T3Bench","owner":"THU-LYJ-Lab","description":"T3Bench: Benchmarking Current Progress in Text-to-3D Generation","archived":false,"fork":false,"pushed_at":"2023-10-24T07:05:13.000Z","size":12191,"stargazers_count":1098,"open_issues_count":5,"forks_count":10,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T18:14:22.572Z","etag":null,"topics":["3d","diffusion","nerf","text-to-3d"],"latest_commit_sha":null,"homepage":"https://t3bench.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THU-LYJ-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-04T13:40:09.000Z","updated_at":"2025-03-24T16:06:02.000Z","dependencies_parsed_at":"2023-10-15T09:31:08.622Z","dependency_job_id":null,"html_url":"https://github.com/THU-LYJ-Lab/T3Bench","commit_stats":null,"previous_names":["thu-lyj-lab/t3bench"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THU-LYJ-Lab%2FT3Bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THU-LYJ-Lab%2FT3Bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THU-LYJ-Lab%2FT3Bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THU-LYJ-Lab%2FT3Bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THU-LYJ-Lab","download_url":"https://codeload.github.com/THU-LYJ-Lab/T3Bench/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248601101,"owners_count":21131607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","diffusion","nerf","text-to-3d"],"created_at":"2024-09-24T20:08:08.111Z","updated_at":"2025-04-12T16:48:44.063Z","avatar_url":"https://github.com/THU-LYJ-Lab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://t3bench.com/static/images/logobig.png\" align=\"center\"\u003e\n\u003cp align=\"center\"\u003e\n    📃 \u003ca href=\"https://arxiv.org/abs/2310.02977\" target=\"_blank\"\u003ePaper\u003c/a\u003e • 🌐 \u003ca href=\"https://t3bench.com\" target=\"_blank\"\u003eProject Page\u003c/a\u003e\n\u003c/p\u003e\n\n# T\u003csup\u003e3\u003c/sup\u003eBench: Benchmarking Current Progress in Text-to-3D Generation\n\n![](fig/A_cactus_with_pink_flowers.gif)\n\n**T\u003csup\u003e3\u003c/sup\u003eBench** is the first comprehensive text-to-3D benchmark containing diverse text prompts of three increasing complexity levels that are specially designed for 3D generation (300 prompts in total). To assess both the subjective quality and the text alignment, we propose two automatic metrics based on multi-view images produced by the 3D contents. The *quality* metric combines multi-view text-image scores and regional convolution to detect quality and view inconsistency. The *alignment* metric uses multi-view captioning and Large Language Model (LLM) evaluation to measure text-3D consistency. Both metrics closely correlate with different dimensions of human judgments, providing a paradigm for efficiently evaluating text-to-3D models.\n\n\u003cimg src=\"https://t3bench.com/static/images/pipeline_v2.png\"\u003e\n\n\n\n## 🔥 Updates\n**[2023/10/24]** We have released mesh results of all prompt sets and methods! Please check \u003ca href=\"https://drive.google.com/file/d/127Pfy6WI8txJU1DjmdpkR2y8-u5DoPIK/view?usp=share_link\"\u003ehere\u003c/a\u003e to download.\n\n\n\n## Evaluate on T\u003csup\u003e3\u003c/sup\u003eBench\n\n### Environment Setup\n\nWe adopt the implementation of \u003ca href=\"https://github.com/threestudio-project/threestudio\"\u003eThreeStudio\u003c/a\u003e to test the current text-to-3D methods. Please first follow the instructions of ThreeStudio to setup the generation environment.\n\nThen install the following packages used for evaluation:\n\n```shell\npip install -r requirements.txt\n```\n\nNote that we use a slightly modified version of ThreeStudio to ensure efficient generation.\n\n\n\n### Evaluation\n\n##### Run Text-to-3D and Extract Mesh\n\n```shell\n# YOUR_GROUP: Choose the prompt set to test, including [single, surr, multi]\n# YOUR_METHOD: We now support latentnerf, magic3d, fantasia3d, dreamfusion, sjc, and prolificdreamer.\npython run_t3.py --group YOUR_GROUP --gpu YOUR_GPU --method YOUR_METHOD\npython run_mesh.py --group YOUR_GROUP --gpu YOUR_GPU --method YOUR_METHOD\n```\n\n\n\n##### Quality Evaluation\n\n```shell\npython run_eval_quality.py --group YOUR_GROUP --gpu YOUR_GPU --method YOUR_METHOD\n```\n\n\n\n##### Alignment Evaluation\n\n```shell\n# First get the 3D prompt of the text-to-3D result\npython run_caption.py --group YOUR_GROUP --gpu YOUR_GPU --method YOUR_METHOD\n# then run the LLM Evaluation\npython run_eval_alignment.py --group YOUR_GROUP --gpu YOUR_GPU --method YOUR_METHOD\n```\n\n\n\n### Citation\n\n```\n@misc{he2023t3bench,\n      title={T$^3$Bench: Benchmarking Current Progress in Text-to-3D Generation}, \n      author={Yuze He and Yushi Bai and Matthieu Lin and Wang Zhao and Yubin Hu and Jenny Sheng and Ran Yi and Juanzi Li and Yong-Jin Liu},\n      year={2023},\n      eprint={2310.02977},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n\n\n### Acknowledgement\n\nThis project could not be possible without the open-source works from \u003ca href=\"https://github.com/threestudio-project/threestudio\"\u003eThreeStudio\u003c/a\u003e, \u003ca href=\"https://github.com/crockwell/Cap3D\"\u003eCap3D\u003c/a\u003e, \u003ca href=\"https://github.com/ashawkey/stable-dreamfusion\"\u003eStable-DreamFusion\u003c/a\u003e, \u003ca href=\"https://github.com/THUDM/ImageReward\"\u003eImageReward\u003c/a\u003e, \u003ca href=\"https://github.com/salesforce/LAVIS\"\u003eLAVIS\u003c/a\u003e. We sincerely thank them all.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthu-lyj-lab%2Ft3bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthu-lyj-lab%2Ft3bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthu-lyj-lab%2Ft3bench/lists"}