Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/atanikan/llm-inference-service

This repo hosts the different ways to run vllm on ANL HPC system
https://github.com/atanikan/llm-inference-service

Last synced: 11 days ago
JSON representation

This repo hosts the different ways to run vllm on ANL HPC system

Awesome Lists containing this project

README

        

# LLM Inference Service
This repository provides the installation and usage of various LLM Inference frameworks on Polaris. The frameworks currently tested are:
* [llama.cpp](llama-cpp/polaris/README.md)
* [vLLM](vllm/polaris/README.md)
* [ollama](ollama/polaris/README.md)
* [deepspeed-inference](deepspeed-inference-mii)