Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/raffaeleterribile/vllmchat
https://github.com/raffaeleterribile/vllmchat
Last synced: 3 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/raffaeleterribile/vllmchat
- Owner: raffaeleterribile
- Created: 2024-10-29T18:44:44.000Z (21 days ago)
- Default Branch: main
- Last Pushed: 2024-10-29T19:05:48.000Z (21 days ago)
- Last Synced: 2024-10-29T20:45:06.176Z (21 days ago)
- Language: Python
- Size: 9.77 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# vLLM Chat
Progetto di prova per studiare vLLM
Avviare il server vLLM con:
vllm serve facebook/opt-125m
Esempi di utilizzo con curl
Elenco dei modelli
curl http://localhost:8000/v1/modelsEsempio di completamento
curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "facebook/opt-125m",
"prompt": "San Francisco is a",
"max_tokens": 7,
"temperature": 0
}'Esempio di chat
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen2.5-1.5B-Instruct",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"}
]
}'