https://github.com/netease-media/grps_trtllm

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

ai-agent chatglm deepseek-r1 function-call internvideo internvl2 janus-pro llama-index llama3 llm minicpm-v multi-modal olmocr openai phi qwen2 qwen2-vl qwq tensorrt-llm

Last synced: 06 Apr 2025