{"id":26538170,"url":"https://github.com/nvidia-cosmos/cosmos-predict1","last_synced_at":"2026-01-18T02:13:54.870Z","repository":{"id":283171215,"uuid":"941557796","full_name":"nvidia-cosmos/cosmos-predict1","owner":"nvidia-cosmos","description":"Cosmos-Predict1 is a collection of general-purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream applications.","archived":false,"fork":false,"pushed_at":"2026-01-06T01:34:31.000Z","size":118581,"stargazers_count":393,"open_issues_count":13,"forks_count":76,"subscribers_count":8,"default_branch":"main","last_synced_at":"2026-01-06T23:02:31.131Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://research.nvidia.com/labs/dir/cosmos-predict1","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvidia-cosmos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-02T15:23:18.000Z","updated_at":"2026-01-06T07:46:07.000Z","dependencies_parsed_at":"2025-08-06T03:18:33.824Z","dependency_job_id":null,"html_url":"https://github.com/nvidia-cosmos/cosmos-predict1","commit_stats":null,"previous_names":["nvidia-cosmos/cosmos-predict1"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nvidia-cosmos/cosmos-predict1","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-predict1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-predict1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-predict1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-predict1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvidia-cosmos","download_url":"https://codeload.github.com/nvidia-cosmos/cosmos-predict1/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-predict1/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28526569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-21T23:01:31.856Z","updated_at":"2026-01-18T02:13:54.856Z","avatar_url":"https://github.com/nvidia-cosmos.png","language":"Jupyter Notebook","funding_links":[],"categories":["🎮 エージェントシミュレーションと世界モデル","Other Related Resources"],"sub_categories":["自動運転","World Foundation Model Platform"],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"assets/nvidia-cosmos-header.png\" alt=\"NVIDIA Cosmos Header\"\u003e\n\u003c/p\u003e\n\n### [Website](https://www.nvidia.com/en-us/ai/cosmos/) | [Hugging Face](https://huggingface.co/collections/nvidia/cosmos-predict1-67c9d1b97678dbf7669c89a7) | [Paper](https://arxiv.org/abs/2501.03575) | [Paper Website](https://research.nvidia.com/labs/dir/cosmos-predict1)\n\n[NVIDIA Cosmos](https://www.nvidia.com/cosmos/) is a developer-first world foundation model platform designed to help Physical AI developers build their Physical AI systems better and faster. Cosmos contains\n\n1. Pre-trained models (available via Hugging Face) under the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) that allows commercial use of the models for free.\n2. Training scripts under the [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0) for post-training the models for various downstream Physical AI applications.\n\n\u003c!-- ------------------------------ --\u003e\n\n## Key Features\n\nCosmos-Predict1 includes the following features:\n\n- **Diffusion-based world foundation models** for Text2World and Video2World generation, where a user can generate visual simulation based on text prompts and video prompts.\n- **Autoregressive-based world foundation models** for Video2World generation, where a user can generate visual simulation based on video prompts and optional text prompts.\n- **Image and video tokenizers** for tokenizing videos into continuous tokens (latent vectors) and discrete tokens (integers) efficiently and effectively.\n\n\u003c!-- ------------------------------ --\u003e\n\n## Examples\n\nInference with pre-trained models:\n* [Inference with diffusion-based Text2World models](examples/inference_diffusion_text2world.md) **[with multi-GPU support]**\n* [Inference with diffusion-based Video2World models](examples/inference_diffusion_video2world.md) **[with multi-GPU support]**\n* [Inference with autoregressive-based base models](examples/inference_autoregressive_base.md) **[with multi-GPU support]**\n* [Inference with autoregressive-based Video2World models](examples/inference_autoregressive_video2world.md) **[with multi-GPU support]**\n* [Inference with tokenizer models](examples/inference_tokenizer.md)\n\nPost-training models:\n* [Post-training diffusion-based Text2World models](examples/post-training_diffusion_text2world.md) **[with multi-GPU support]**\n* [Post-training diffusion-based Video2World models](examples/post-training_diffusion_video2world.md) **[with multi-GPU support]**\n* [Post-training diffusion-based Text2World models (with multi-view data)](examples/post-training_diffusion_text2world_multiview.md) **[with multi-GPU support]**\n* [Post-training diffusion-based Video2World models (with multi-view data)](examples/post-training_diffusion_video2world_multiview.md) **[with multi-GPU support]**\n* [Post-training autoregressive-based base models](examples/post-training_autoregressive_base.md) **[with multi-GPU support]**\n* [Post-training tokenizer models](examples/post-training_tokenizer.md) **[with multi-GPU support]**\n\nInference with post-trained models:\n* [Inference with diffusion-based Text2World models (with multi-view data)](examples/inference_diffusion_text2world_multiview.md) **[with multi-GPU support]**\n* [Inference with diffusion-based Video2World models (with multi-view data)](examples/inference_diffusion_video2world_multiview.md) **[with multi-GPU support]**\n\n\nThe code snippet below provides a gist of the inference usage.\n\n```bash\nPROMPT=\"A sleek, humanoid robot stands in a vast warehouse filled with neatly stacked cardboard boxes on industrial shelves. The robot's metallic body gleams under the bright, even lighting, highlighting its futuristic design and intricate joints. A glowing blue light emanates from its chest, adding a touch of advanced technology. The background is dominated by rows of boxes, suggesting a highly organized storage system. The floor is lined with wooden pallets, enhancing the industrial setting. The camera remains static, capturing the robot's poised stance amidst the orderly environment, with a shallow depth of field that keeps the focus on the robot while subtly blurring the background for a cinematic effect.\"\n\nCUDA_HOME=$CONDA_PREFIX PYTHONPATH=$(pwd) python cosmos_predict1/diffusion/inference/text2world.py \\\n    --checkpoint_dir checkpoints \\\n    --diffusion_transformer_dir Cosmos-Predict1-7B-Text2World \\\n    --prompt \"${PROMPT}\" \\\n    --offload_prompt_upsampler \\\n    --video_save_name diffusion-text2world-7b\n```\n\n\u003cvideo src=\"https://github.com/user-attachments/assets/2ee7386b-8808-4db2-b38a-87ab679339f9\"\u003e\n  Your browser does not support the video tag.\n\u003c/video\u003e\n\n\n\u003c!-- ------------------------------ --\u003e\n\n## Model Family\n\nWe provide a series of pre-trained models of different families, available for download on Hugging Face.\n\n**Diffusion models**\n\n* [Cosmos-Predict1-7B-Text2World](https://huggingface.co/nvidia/Cosmos-Predict1-7B-Text2World): Text to visual world generation\n* [Cosmos-Predict1-14B-Text2World](https://huggingface.co/nvidia/Cosmos-Predict1-14B-Text2World): Text to visual world generation\n* [Cosmos-Predict1-7B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict1-7B-Video2World): Video + Text based future visual world generation\n* [Cosmos-Predict1-14B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict1-14B-Video2World): Video + Text based future visual world generation\n\n**Autoregressive models**\n\n* [Cosmos-Predict1-4B](https://huggingface.co/nvidia/Cosmos-Predict1-4B): Future visual world generation\n* [Cosmos-Predict1-12B](https://huggingface.co/nvidia/Cosmos-Predict1-12B): Future visual world generation\n* [Cosmos-Predict1-5B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict1-5B-Video2World): Video + Text based future visual world generation\n* [Cosmos-Predict1-13B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict1-13B-Video2World): Video + Text based future visual world generation\n\n**Tokenizers**\n\n* [Cosmos-Tokenize1-CV8×8×8-720p](https://huggingface.co/nvidia/Cosmos-Tokenize1-CV8x8x8-720p): Continuous Video Tokenizer with 8x8x8 spatio-temporal compression with, 121 frames context\n* [Cosmos-Tokenize1-DV8×16×16-720p](https://huggingface.co/nvidia/Cosmos-Tokenize1-DV8x16x16-720p): Discrete Video Tokenizer with 8x16x16 spatio-temporal compression, and 49 frames context\n* [Cosmos-Tokenize1-CI8×8-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-CI8x8-360p): Continuous Image Tokenizer with 8x8 spatial compression with low-resolution support\n* [Cosmos-Tokenize1-CI16x16-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-CI16x16-360p): Continuous Image Tokenizer with 16x16 spatial compression with low-resolution support\n* [Cosmos-Tokenize1-CV4×8×8-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-CV4x8x8-360p): Continuous Video Tokenizer with 4x8x8 spatio-temporal compression with low-resolution support\n* [Cosmos-Tokenize1-DI8×8-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-DI8x8-360p): Discrete Image Tokenizer with 8x8 spatial compression with low-resolution support\n* [Cosmos-Tokenize1-DI16x16-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-DI16x16-360p): Discrete Image Tokenizer with 16x16 spatial compression with low-resolution support\n* [Cosmos-Tokenize1-DV4×8×8-360p](https://huggingface.co/nvidia/Cosmos-Tokenize1-DV4x8x8-360p): Discrete Video Tokenizer with 4x8x8 spatio-temporal compression with low-resolution support\n\n\u003c!-- ------------------------------ --\u003e\n\n## License and Contact\n\nThis project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.\n\nNVIDIA Cosmos source code is released under the [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0).\n\nNVIDIA Cosmos models are released under the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license). For a custom license, please contact [cosmos-license@nvidia.com](mailto:cosmos-license@nvidia.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-cosmos%2Fcosmos-predict1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvidia-cosmos%2Fcosmos-predict1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-cosmos%2Fcosmos-predict1/lists"}