https://github.com/skyworkai/skywork-r1v
Skywork-R1V2 : Multimodal Hybrid Reinforcement Learning for Reasoning(最好的多模态推理)
https://github.com/skyworkai/skywork-r1v
deepseek-r1 grpo llm multimodal-r1 multimodal-understanding r1v reasoning reinforcement-learning skywork-r1v vlm vlm-r1
Last synced: about 1 month ago
JSON representation
Skywork-R1V2 : Multimodal Hybrid Reinforcement Learning for Reasoning(最好的多模态推理)
- Host: GitHub
- URL: https://github.com/skyworkai/skywork-r1v
- Owner: SkyworkAI
- License: mit
- Created: 2025-03-15T08:11:44.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-04-28T09:46:51.000Z (about 2 months ago)
- Last Synced: 2025-05-05T06:41:10.902Z (about 1 month ago)
- Topics: deepseek-r1, grpo, llm, multimodal-r1, multimodal-understanding, r1v, reasoning, reinforcement-learning, skywork-r1v, vlm, vlm-r1
- Language: Python
- Homepage:
- Size: 36.3 MB
- Stars: 2,405
- Watchers: 142
- Forks: 236
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Skywork-R1V: Pioneering Multimodal Reasoning with CoT
[[🤗 Skywork-R1V2-38B](https://huggingface.co/Skywork/Skywork-R1V2-38B)] [[🤖 R1V2 ModelScope](https://modelscope.cn/models/Skywork/Skywork-R1V2-38B)] [[📖 R1V2 Report](https://arxiv.org/abs/2504.16656)]
[[🤗 Skywork-R1V-38B](https://huggingface.co/Skywork/Skywork-R1V-38B)] [[📖 R1V1 Report](https://arxiv.org/abs/2504.05599)]
Welcome to the Skywork-R1V repository! Here, you'll find the model weights and inference code for our state-of-the-art open-sourced multimodal reasoning model, enabling advanced visual and text thinking.
## 🔥News
**April 28, 2025**: We released awq quantized version of Skywork R1V2[[🤗 Skywork-R1V2-38B-AWQ](https://huggingface.co/Skywork/Skywork-R1V2-38B-AWQ)], supporting single-card (above 30GB) inference.
**April 24, 2025**: We released **Skywork-R1V2**, a state-of-the-art, open-source multimodal reasoning model that achieves leading performance across multiple vision-language benchmarks.[[🤗 Skywork-R1V2-38B](https://huggingface.co/Skywork/Skywork-R1V2-38B)][[📖R1V2 Report](https://arxiv.org/abs/2504.16656)]
**April 9, 2025**: Our technical report is currently available on arxiv: [[Skywork-R1V: Pioneering Multimodal Reasoning with CoT](https://arxiv.org/abs/2504.05599)].
**April 1, 2025**: Skywork-R1V supports inference with [[vLLM](https://github.com/vllm-project/vllm)], On 4×L20Y GPUs, vLLM generates 1k tokens in ~12.3s, at least 5× faster than transformers.
**Mar 26, 2025**: We released awq quantized version of Skywork R1V[[🤗 Skywork-R1V-38B-AWQ](https://huggingface.co/Skywork/Skywork-R1V-38B-AWQ)], supporting single-card (above 30GB) inference.
**Mar 18, 2025**: We are thrilled to introduce Skywork R1V, the first industry open-sourced multimodal reasoning model with advanced visual chain-of-thought capabilities, pushing the boundaries of AI-driven vision and logical inference! 🚀
## R1V2-38B Evaluation
Skywork-R1V2-38B demonstrates state-of-the-art performance on both text and multimodal reasoning tasks.
Comparison of Skywork-R1V2 with Multimodal Open-Source and Proprietary Models
Model
Text Reasoning (pass@1 or %)
Multimodal Reasoning (%)
AIME24
LiveCodebench
liveBench
IFEVAL
BFCL
MMMU(val)
MathVista(mini)
MathVision(mini)
OlympiadBench
mmmu-pro
Skywork-R1V2-38B
78.9
63.6
73.2
82.9
66.3
73.6
74.0
49.0
62.6
52.0
OpenAI-4o
74.6
9.3
49.9
—
—
69.1
63.8
58.0
—
—
Claude 3.5 Sonnet
16.0
—
65.0
—
—
66.4
65.3
—
—
—
Kimi k1.5
77.5
—
—
—
—
70.0
74.9
—
—
—
Qwen2.5-VL-72B
—
—
—
—
—
70.2
74.8
38.1
40.4
—
InternVL3-38B
—
—
—
—
—
70.1
75.1
34.2
-
—
Text Reasoning Performance
Multimodal Reasoning vs Proprietary Models
Multimodal Reasoning vs Open-Source Models
## How to Run Locally
### 1. Clone the Repository
```shell
git clone https://github.com/SkyworkAI/Skywork-R1V.git
cd skywork-r1v/inference
```
### 2. Set Up the Environment
```shell
# For Transformers
conda create -n r1-v python=3.10 && conda activate r1-v
bash setup.sh
# For vLLM
conda create -n r1v-vllm python=3.10 && conda activate r1v-vllm
pip install -U vllm
```
### 3. Run the Inference Script
#### Using Transformers
```shell
CUDA_VISIBLE_DEVICES="0,1" python inference_with_transformers.py \
--model_path path \
--image_paths image1_path \
--question "your question"
```
#### Using vLLM
```shell
python inference_with_vllm.py \
--model_path path \
--image_paths image1_path image2_path \
--question "your question" \
--tensor_parallel_size 4
```
## License
This code repository is licensed under [the MIT License](https://github.com/SkyworkAI/Skywork-R1V/blob/main/LICENSE).
✅ Commercial use permitted
✅ Modification allowed
✅ Distribution allowed
❌ No liability
## Citation
If you use Skywork-R1V in your research, please cite:
```
@misc{chris2025skyworkr1v2multimodalhybrid,
title={Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning},
author={Chris and Yichen Wei and Yi Peng and Xiaokun Wang and Weijie Qiu and Wei Shen and Tianyidan Xie and Jiangbo Pei and Jianhao Zhang and Yunzhuo Hao and Xuchen Song and Yang Liu and Yahui Zhou},
year={2025},
eprint={2504.16656},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2504.16656},
}
```
```
@misc{peng2025skyworkr1vpioneeringmultimodal,
title={Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought},
author={Yi Peng and Chris and Xiaokun Wang and Yichen Wei and Jiangbo Pei and Weijie Qiu and Ai Jian and Yunzhuo Hao and Jiachun Pan and Tianyidan Xie and Li Ge and Rongxian Zhuang and Xuchen Song and Yang Liu and Yahui Zhou},
year={2025},
eprint={2504.05599},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2504.05599},
}
```