{"id":26741886,"url":"https://github.com/Roblox/cube","last_synced_at":"2025-03-28T06:02:45.213Z","repository":{"id":283385719,"uuid":"946920596","full_name":"Roblox/cube","owner":"Roblox","description":"Roblox Foundation Model for 3D Intelligence","archived":false,"fork":false,"pushed_at":"2025-03-20T00:33:38.000Z","size":25687,"stargazers_count":13,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-20T01:22:51.356Z","etag":null,"topics":["3d","3d-aigc","3d-generation","shape-generation","text-to-3d"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Roblox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-11T22:09:47.000Z","updated_at":"2025-03-20T00:33:42.000Z","dependencies_parsed_at":"2025-03-20T01:37:46.643Z","dependency_job_id":null,"html_url":"https://github.com/Roblox/cube","commit_stats":null,"previous_names":["roblox/cube"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roblox%2Fcube","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roblox%2Fcube/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roblox%2Fcube/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roblox%2Fcube/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Roblox","download_url":"https://codeload.github.com/Roblox/cube/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245978266,"owners_count":20703677,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","3d-aigc","3d-generation","shape-generation","text-to-3d"],"created_at":"2025-03-28T06:01:50.717Z","updated_at":"2025-03-28T06:02:45.203Z","avatar_url":"https://github.com/Roblox.png","language":"Python","funding_links":[],"categories":["3D视觉生成重建","Related Resources"],"sub_categories":["资源传输下载","Survey and Awesome Repos"],"readme":"# Cube: Generative AI System for 3D\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./resources/teaser.png\" width=\"800\" style=\"margin: 5px;\"\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=https://corp.roblox.com/newsroom/2025/03/introducing-roblox-cube target=\"_blank\"\u003e\u003cimg src=https://img.shields.io/badge/Roblox-Blog-000000.svg?logo=Roblox height=22px\u003e\u003c/a\u003e\n  \u003ca href=https://arxiv.org/abs/2503.15475 target=\"_blank\"\u003e\u003cimg src=https://img.shields.io/badge/ArXiv-Report-b5212f.svg?logo=arxiv height=22px\u003e\u003c/a\u003e\n  \u003ca href=https://huggingface.co/Roblox/cube3d-0.1 target=\"_blank\"\u003e\u003cimg src=https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Models-d96902.svg height=22px\u003e\u003c/a\u003e\n  \u003ca href=https://huggingface.co/spaces/Roblox/cube3d-interactive target=\"_blank\"\u003e\u003cimg src=https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Demo-blue.svg height=22px\u003e\u003c/a\u003e\n  \u003ca href=https://colab.research.google.com/drive/1ZvTj49pjDCD_crX5WPZNTAoTTzL6-E5t target=\"_blank\"\u003e\u003cimg src=https://img.shields.io/badge/Colab-Demo-blue.svg?logo=googlecolab height=22px\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n\nFoundation models trained on vast amounts of data have demonstrated remarkable reasoning and\ngeneration capabilities in the domains of text, images, audio and video. Our goal is to build\nsuch a foundation model for 3D intelligence, a model that can support developers in producing all aspects\nof a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to\nproducing programmatic scripts describing object behaviors. As we start open-sourcing a family of models \ntowards this vision, we hope to engage others in the research community to address these goals with us.\n\n## Get Started with Cube 3D\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./resources/greyscale_512.gif\" width=\"600\" style=\"margin: 5px;\"\u003e\n\u003c/p\u003e\n\nCube 3D is our first step towards 3D intelligence, which involves a shape tokenizer and a text-to-shape generation model. We are unlocking the power of generating 3D assets and enhancing creativity for all artists. Our latest version of Cube 3D is now accessible to individuals, creators, researchers and businesses of all sizes so that they can experiment, innovate and scale their ideas responsibly. This release includes model weights and starting code for using our text-to-shape model to create 3D assets.\n\n### Try it out on \n\n- [Google Colab](https://colab.research.google.com/drive/1ZvTj49pjDCD_crX5WPZNTAoTTzL6-E5t)\n- [Hugging Face Interactive Demo](https://huggingface.co/spaces/Roblox/cube3d-interactive)\n\n### Install Requirements\n\nClone and install this repo in a virtual environment, via:\n\n```bash\ngit clone https://github.com/Roblox/cube.git\ncd cube\npip install -e .[meshlab]\n```\n\n\u003e **CUDA**: If you are using a Windows machine, you may need to install the [CUDA](https://developer.nvidia.com/cuda-downloads) toolkit as well as `torch` with cuda support via `pip install torch --index-url https://download.pytorch.org/whl/cu124 --force-reinstall`\n\n\u003e **Note**: `[meshlab]` is an optional dependency and can be removed by simply running `pip install -e .` for better compatibility but mesh simplification will be disabled.\n\n### Download Models from Huggingface 🤗\n\nDownload the model weights from [hugging face](https://huggingface.co/Roblox/cube3d-v0.1) or use the\n`huggingface-cli`:\n\n```bash\nhuggingface-cli download Roblox/cube3d-v0.1 --local-dir ./model_weights\n```\n\n### Inference\n\n#### 1. Shape Generation\n\nTo generate 3D models using the downloaded models simply run:\n\n```bash\npython -m cube3d.generate \\\n            --gpt-ckpt-path model_weights/shape_gpt.safetensors \\\n            --shape-ckpt-path model_weights/shape_tokenizer.safetensors \\\n            --fast-inference \\\n            --prompt \"Broad-winged flying red dragon, elongated, folded legs.\"\n```\n\n\u003e **Note**: `--fast-inference` is optional and may not be available for all GPU that have limited VRAM. This flag will also not work on MacOS. \n\nThe output will be an `.obj` file saved in the specified `output` directory.\n\nIf you want to render a turntable gif of the mesh, you can use the `--render-gif` flag, which will render a turntable gif of the mesh\nand save it as `turntable.gif` in the specified `output` directory. \n\nWe provide several example output objects and their corresponding text prompts in the `examples` folder.\n\n\u003e **Note**: You must have Blender (version \u003e= 4.3) installed and available in your system's PATH to render the turntable GIF. You can download it from [Blender's official website](https://www.blender.org/). Ensure that the Blender executable is accessible from the command line.\n\n\u003e **Note**: If shape decoding is slow, you can try to specify a lower resolution using the `--resolution-base` flag. A lower resolution will create a coarser and lower quality output mesh but faster decoding. Values between 4.0 and 9.0 are recommended.\n\n#### 2. Shape Tokenization and De-tokenization\n\nTo tokenize a 3D shape into token indices and reconstruct it back, you can use the following command:\n\n```bash\npython -m cube3d.vq_vae_encode_decode \\\n            --shape-ckpt-path model_weights/shape_tokenizer.safetensors \\\n            --mesh-path ./outputs/output.obj\n```\n\nThis will process the `.obj` file located at `./outputs/output.obj` and prints the tokenized representation as well as exports the mesh reconstructed from the token indices.\n\n### Hardware Requirements\n\nWe have tested our model on:\n* Nvidia H100 GPU\n* Nvidia A100 GPU\n* Nvidia Geforce 3080\n* Apple Silicon M2-4 Chips.\n\nWe recommend using a GPU with at least 24GB of VRAM available when using `--fast-inference` (or `EngineFast`) and 16GB otherwise. \n\n### Code Usage\n\nWe have designed a minimalist API that allows the use this repo as a Python library:\n\n```python\nimport torch\nimport trimesh\nfrom cube3d.inference.engine import Engine, EngineFast\n\n# load ckpt\nconfig_path = \"cube3d/configs/open_model.yaml\"\ngpt_ckpt_path = \"model_weights/shape_gpt.safetensors\"\nshape_ckpt_path = \"model_weights/shape_tokenizer.safetensors\"\nengine_fast = EngineFast( # only supported on CUDA devices, replace with Engine otherwise\n    config_path, \n    gpt_ckpt_path, \n    shape_ckpt_path, \n    device=torch.device(\"cuda\"),\n)\n\n# inference\ninput_prompt = \"A pair of noise-canceling headphones\"\n# NOTE: Reduce `resolution_base` for faster inference and lower VRAM usage\n# The `top_p` parameter controls randomness between inferences:\n#   Float \u003c 1: Keep smallest set of tokens with cumulative probability ≥ top_p. Default None: deterministic generation.\nmesh_v_f = engine_fast.t2s([input_prompt], use_kv_cache=True, resolution_base=8.0, top_p=0.9)\n\n# save output\nvertices, faces = mesh_v_f[0][0], mesh_v_f[0][1]\n_ = trimesh.Trimesh(vertices=vertices, faces=faces).export(\"output.obj\")\n```\n\n## Coming Soon\n\n### Controlling shape generation with bounding box conditioning\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./resources/truck_black_text_512.gif\" width=\"300\" height=\"300\" style=\"margin: 5px;\"\u003e\n  \u003cimg src=\"./resources/couch_black_text_512.gif\" width=\"300\" height=\"300\" style=\"margin: 5px;\"\u003e\n\u003c/div\u003e\n\n### Scene Generation\n\nhttps://github.com/user-attachments/assets/987c459a-5708-41a5-9b92-89068a70a239\n\nhttps://github.com/user-attachments/assets/ab501a86-b0cb-4c73-827e-988b2120d4c0\n\n## Citation\nIf you find this work helpful, please consider citing our technical report:\n\n```bibtex\n@article{roblox2025cube,\n    title   = {Cube: A Roblox View of 3D Intelligence},\n    author  = {Roblox, Foundation AI Team},\n    journal = {arXiv preprint arXiv:2503.15475},\n    year    = {2025}\n}\n```\n\n## Acknowledgements\n\nWe would like to thank the contributors of [TRELLIS](https://github.com/microsoft/TRELLIS), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D), [threestudio](https://github.com/threestudio-project/threestudio), [Hunyuan3D-2](https://github.com/Tencent/Hunyuan3D-2), [minGPT](https://github.com/karpathy/minGPT), [dinov2](https://github.com/facebookresearch/dinov2), [OptVQ](https://github.com/zbr17/OptVQ), [1d-tokenizer](https://github.com/bytedance/1d-tokenizer)\nrepositories, for their open source contributions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRoblox%2Fcube","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRoblox%2Fcube","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRoblox%2Fcube/lists"}