https://github.com/dzooli/deepseek-vl
A simple API server to use Deepseek-VL for image description generation.
https://github.com/dzooli/deepseek-vl
Last synced: about 2 months ago
JSON representation
A simple API server to use Deepseek-VL for image description generation.
- Host: GitHub
- URL: https://github.com/dzooli/deepseek-vl
- Owner: dzooli
- License: mit
- Created: 2025-08-23T16:03:20.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2025-08-23T17:44:17.000Z (about 2 months ago)
- Last Synced: 2025-08-23T23:45:10.971Z (about 2 months ago)
- Language: Python
- Size: 12.4 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE-CODE
Awesome Lists containing this project
README
## DeepSeek-VL FastAPI Service
This repository includes a FastAPI-based vision-language inference service in `app.py` . It is based on DeepSeek-VL.
### Description
`app.py` provides a REST API endpoint for multimodal (image + text) inference using DeepSeek-VL models. It loads a vision-language model and exposes a `/v1/vision-chat` endpoint that accepts an image and a prompt, returning the model's response. The service is designed for fast, real-world vision-language understanding tasks.
**Key features:**
* Loads DeepSeek-VL models (default: `deepseek-ai/deepseek-vl-7b-chat`)
* Accepts image and text prompt via HTTP POST
* Returns model-generated response
* Optimized for speed and memory usage
### Installation
Ensure you have Python >= 3.8. Install dependencies:
```bash
pip install -e .
# or
uv sync
```
### Running the FastAPI Service
You can launch the API server using [uvicorn](https://www.uvicorn.org/):
```bash
uvicorn app:app --port 8901
```
### Example API Usage
Send an image and prompt to the API using `curl` :
```bash
curl -X POST "http://localhost:8901/v1/vision-chat" \
-F "prompt=What is in this image?" \
-F "image=@/path/to/your/image.png"
```
The response will be a JSON object with the model's output.
---
# --- Original README.md starts here ---
---
📥 Model Download |
⚡ Quick Start |
📜 License |
📖 Citation
📄 Paper Link |
🤗 Huggingface Paper Link |
👁️ Demo
## 1. Introduction
Introducing DeepSeek-VL, an open-source Vision-Language (VL) Model designed for real-world vision and language understanding applications. DeepSeek-VL possesses general multimodal understanding capabilities, capable of processing logical diagrams, web pages, formula recognition, scientific literature, natural images, and embodied intelligence in complex scenarios.
[DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://arxiv.org/abs/2403.05525)
Haoyu Lu*, Wen Liu*, Bo Zhang**, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, Chong Ruan (*Equal Contribution, **Project Lead)
## 2. Release
✅ 2024-03-14: Demo for DeepSeek-VL-7B available on Hugging Face.
Check out the gradio demo of DeepSeek-VL-7B at https://huggingface.co/spaces/deepseek-ai/DeepSeek-VL-7B. Experience its capabilities firsthand!
✅ 2024-03-13: Support DeepSeek-VL gradio demo.
✅ 2024-03-11: DeepSeek-VL family released, including DeepSeek-VL-7B-base, DeepSeek-VL-7B-chat, DeepSeek-VL-1.3B-base, and DeepSeek-VL-1.3B-chat.
The release includes a diverse set of models tailored for various applications within the DeepSeek-VL family. The models come in two sizes: 7B and 1.3B parameters, each offering base and chat variants to cater to different needs and integration scenarios.
## 3. Model Downloads
We release the DeepSeek-VL family, including 1.3B-base, 1.3B-chat, 7b-base and 7b-chat models, to the public.
To support a broader and more diverse range of research within both academic and commercial communities.
Please note that the use of this model is subject to the terms outlined in [License section](#5-license). Commercial usage is
permitted under these terms.
### Huggingface
| Model | Sequence Length | Download |
|-----------------------|-----------------|-----------------------------------------------------------------------------|
| DeepSeek-VL-1.3B-base | 4096 | [🤗 Hugging Face](https://huggingface.co/deepseek-ai/deepseek-vl-1.3b-base) |
| DeepSeek-VL-1.3B-chat | 4096 | [🤗 Hugging Face](https://huggingface.co/deepseek-ai/deepseek-vl-1.3b-chat) |
| DeepSeek-VL-7B-base | 4096 | [🤗 Hugging Face](https://huggingface.co/deepseek-ai/deepseek-vl-7b-base) |
| DeepSeek-VL-7B-chat | 4096 | [🤗 Hugging Face](https://huggingface.co/deepseek-ai/deepseek-vl-7b-chat) |
## 4. Quick Start
### Installation
On the basis of `Python >= 3.8` environment, install the necessary dependencies by running the following command:
```shell
pip install -e .
```
### Simple Inference Example
```python
import torch
from transformers import AutoModelForCausalLM
from deepseek_vl.models import VLChatProcessor, MultiModalityCausalLM
from deepseek_vl.utils.io import load_pil_images
# specify the path to the model
model_path = "deepseek-ai/deepseek-vl-7b-chat"
vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer
vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()
## single image conversation example
conversation = [
{
"role": "User",
"content": "Describe each stage of this image.",
"images": ["./images/training_pipelines.jpg"],
},
{"role": "Assistant", "content": ""},
]
## multiple images (or in-context learning) conversation example
# conversation = [
# {
# "role": "User",
# "content": "A dog wearing nothing in the foreground, "
# "a dog wearing a santa hat, "
# "a dog wearing a wizard outfit, and "
# "what's the dog wearing?",
# "images": [
# "images/dog_a.png",
# "images/dog_b.png",
# "images/dog_c.png",
# "images/dog_d.png",
# ],
# },
# {"role": "Assistant", "content": ""}
# ]
# load images and prepare for inputs
pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(
conversations=conversation,
images=pil_images,
force_batchify=True
).to(vl_gpt.device)
# run image encoder to get the image embeddings
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)
# run the model to get the response
outputs = vl_gpt.language_model.generate(
inputs_embeds=inputs_embeds,
attention_mask=prepare_inputs.attention_mask,
pad_token_id=tokenizer.eos_token_id,
bos_token_id=tokenizer.bos_token_id,
eos_token_id=tokenizer.eos_token_id,
max_new_tokens=512,
do_sample=False,
use_cache=True
)
answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True)
print(f"{prepare_inputs['sft_format'][0]}", answer)
```
### CLI Chat
```bash
python cli_chat.py --model_path "deepseek-ai/deepseek-vl-7b-chat"
# or local path
python cli_chat.py --model_path "local model path"
```
### Gradio Demo
```bash
pip install -e .[gradio]
python deepseek_vl/serve/app_deepseek.py
```
Have Fun!
## 5. License
This code repository is licensed under [the MIT License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-CODE). The use of DeepSeek-VL Base/Chat models is subject to [DeepSeek Model License](https://github.com/deepseek-ai/DeepSeek-LLM/blob/HEAD/LICENSE-MODEL). DeepSeek-VL series (including Base and Chat) supports commercial use.
## 6. Citation
```
@misc{lu2024deepseekvl,
title={DeepSeek-VL: Towards Real-World Vision-Language Understanding},
author={Haoyu Lu and Wen Liu and Bo Zhang and Bingxuan Wang and Kai Dong and Bo Liu and Jingxiang Sun and Tongzheng Ren and Zhuoshu Li and Hao Yang and Yaofeng Sun and Chengqi Deng and Hanwei Xu and Zhenda Xie and Chong Ruan},
year={2024},
eprint={2403.05525},
archivePrefix={arXiv},
primaryClass={cs.AI}
}
```
## 7. Contact
If you have any questions, please raise an issue or contact us at [service@deepseek.com](mailto:service@deepseek.com).
