{"id":26538171,"url":"https://github.com/nvidia-cosmos/cosmos-reason1","last_synced_at":"2026-01-18T02:11:59.981Z","repository":{"id":283172702,"uuid":"941558026","full_name":"nvidia-cosmos/cosmos-reason1","owner":"nvidia-cosmos","description":"Cosmos-Reason1 models understand the physical common sense and generate appropriate embodied decisions in natural language through long chain-of-thought reasoning processes.","archived":false,"fork":false,"pushed_at":"2025-03-18T21:23:47.000Z","size":1342,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-18T23:27:59.570Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvidia-cosmos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-02T15:23:55.000Z","updated_at":"2025-03-18T22:49:07.000Z","dependencies_parsed_at":"2025-03-18T23:38:09.222Z","dependency_job_id":null,"html_url":"https://github.com/nvidia-cosmos/cosmos-reason1","commit_stats":null,"previous_names":["nvidia-cosmos/cosmos-reason1"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-reason1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-reason1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-reason1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvidia-cosmos%2Fcosmos-reason1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvidia-cosmos","download_url":"https://codeload.github.com/nvidia-cosmos/cosmos-reason1/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244880607,"owners_count":20525513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-21T23:01:31.883Z","updated_at":"2026-01-18T02:11:59.974Z","avatar_url":"https://github.com/nvidia-cosmos.png","language":null,"funding_links":[],"categories":["A01_文本生成_文本对话","Other Related Resources"],"sub_categories":["大语言对话模型及数据","World Foundation Model Platform"],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"assets/nvidia-cosmos-header.png\" alt=\"NVIDIA Cosmos Header\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003e\n\n\u003e 🚨 Update Notice\n\u003e\n\u003e The latest version of our Cosmos-Reason is now live!\n\u003e\n\u003e 👉 [**Cosmos-Reason2**](https://github.com/nvidia-cosmos/cosmos-reason2)\n\u003e \n\u003e We recommend all users migrate to the new version for improved performance, features, and continued support.\n\u003c/h1\u003e\n\n### [Paper](https://arxiv.org/abs/2503.15558) | [Website](https://research.nvidia.com/labs/dir/cosmos-reason1/) | [HuggingFace](https://huggingface.co/collections/nvidia/cosmos-reason1-67c9e926206426008f1da1b7) | [Cosmos Cookbook](https://github.com/nvidia-cosmos/cosmos-cookbook)\n\nNVIDIA Cosmos Reason – an open, customizable, 7B-parameter reasoning vision language model (VLM) for physical AI and robotics - enables robots and vision AI agents to reason like humans, using prior knowledge, physics understanding and common sense to understand and act in the real world. This model understands space, time, and fundamental physics, and can serve as a planning model to reason what steps an embodied agent might take next.\nCosmos Reason excels at navigating the long tail of diverse scenarios of the physical world with spatial-temporal understanding. Cosmos Reason is post-trained with physical common sense and embodied reasoning data with supervised fine-tuning and reinforcement learning. It uses chain-of-thought reasoning capabilities to understand world dynamics without human annotations.\n\n## News\n\n* 2025-10-28: We added [Cosmos Cookbook](https://github.com/nvidia-cosmos/cosmos-cookbook), a collection of step-by-step recipes and post-training scripts to quickly build, customize, and deploy NVIDIA’s Cosmos world foundation models for robotics and autonomous systems.\n* 2025-08-08: We added the [`cosmos-reason1-utils`](cosmos_reason1_utils/README.md) inference utilities package. Adds spatial-temporal reasoning inference. See [Inference](#inference) for example usage.\n* 2025-08-1: We added support for spatial-temporal reasoning for city and industrial operations. See latest checkpoint [Cosmos-Reason1-7B](https://huggingface.co/nvidia/Cosmos-Reason1-7B).\n* 2025-06-11: We enhance the model’s capability on judging the physical plausibility of a video. See [this tutorial](examples/video_critic/README.md) for details.\n* 2025-05-17: We release model weights and training data under [Hugging Face](https://huggingface.co/collections/nvidia/cosmos-reason1-67c9e926206426008f1da1b7).\n\n## Model\n\n* [Cosmos-Reason1-7B](https://huggingface.co/nvidia/Cosmos-Reason1-7B)\n\n## Setup\n\n\u003e **This repository only contains documentation/examples/utilities. You do not need it to run inference. See [Inference example](scripts/inference_sample.py) for a minimal inference example. The following setup instructions are only needed to run the examples in this repository.**\n\nInstall system dependencies:\n\n* [uv](https://docs.astral.sh/uv/getting-started/installation/)\n\n  ```shell\n  curl -LsSf https://astral.sh/uv/install.sh | sh\n  source $HOME/.local/bin/env\n  ```\n\n* [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/en/guides/cli)\n\n  ```shell\n  uv tool install -U \"huggingface_hub[cli]\"\n  hf auth login\n  ```\n\nClone the repository:\n\n```shell\ngit clone https://github.com/nvidia-cosmos/cosmos-reason1.git\ncd cosmos-reason1\n```\n\n## Inference\n\nMinimum Requirements:\n\n* 1 GPU with 24GB memory\n\nCosmos-Reason1 is included in [`transformers\u003e=4.51.3`](https://huggingface.co/docs/transformers/en/index).\n\nWe provide example inference scripts:\n\n* [Minimal example](scripts/inference_sample.py)\n\n  ```shell\n  uv run scripts/inference_sample.py\n  ```\n\n* [Full example](scripts/inference.py)\n\n  Caption the video:\n\n  ```shell\n  ./scripts/inference.py --prompt prompts/caption.yaml --videos assets/sample.mp4 -v\n  ```\n\n  Ask a question about the video with reasoning:\n\n  ```shell\n  ./scripts/inference.py --prompt prompts/question.yaml --question 'What are the potential safety hazards?' --reasoning --videos assets/sample.mp4 -v\n  ```\n\n  Temporally caption the video and save the input frames to `outputs/temporal_caption_text` for debugging:\n\n  ```shell\n  ./scripts/inference.py --prompt prompts/temporal_caption_text.yaml --videos assets/sample.mp4 --timestamp -v -o outputs/temporal_caption_text\n  ```\n\n  Configure inference by editing:\n\n  * [Prompts](prompts/README.md)\n  * [Sampling Parameters](configs/sampling_params.yaml)\n  * [Vision Processor Config](configs/vision_config.yaml)\n\n## Tutorials\n\n* [Video Critic](examples/video_critic/README.md)\n* Post-Training\n  * [Full example](examples/post_training/README.md)\n  * [Minimal Hugging Face example](examples/post_training_hf/README.md)\n  * [Minimal Llava example](examples/post_training_llava/README.md)\n* [Benchmark](examples/benchmark/README.md)\n\n## Post-Training\n\nThe [nvidia-cosmos/cosmos-rl](https://github.com/nvidia-cosmos/cosmos-rl) repository is an async post-training framework specialized for Supervised Fine-Tuning (SFT) and Reinforcement Learning with Human Feedback (RLHF). It prioritizes performance, scalability, and fault tolerance.\n\nTo support a custom dataset format, use the [minimal Hugging Face example](examples/post_training_hf/README.md) as a template.\n\n## Additional Resources\n\nThe Cosmos-Reason1 model is based on the Qwen2.5-VL model architecture. Useful resources:\n\n* [Repository](https://github.com/QwenLM/Qwen2.5-VL/blob/main/README.md)\n\n## Post-Training quantization\n\nTo run PTQ `\"vllm==0.9.2\" \"transformers\u003e=4.53.1\" \"qwen-vl-utils[decord]\" \"llmcompressor\u003e=0.6.0\"` are required\n\n[Full example](scripts/quantize_fp8.py)\n\n```shell\n./scripts/quantize_fp8.py --model_id 'nvidia/Cosmos-Reason1-7B' --save_dir 'Cosmos-Reason1-7B-W8A8-FP8'\n```\n\n## License and Contact\n\nThis project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.\n\nNVIDIA Cosmos source code is released under the [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0).\n\nNVIDIA Cosmos models are released under the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license). For a custom license, please contact [cosmos-license@nvidia.com](mailto:cosmos-license@nvidia.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-cosmos%2Fcosmos-reason1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvidia-cosmos%2Fcosmos-reason1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvidia-cosmos%2Fcosmos-reason1/lists"}