Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Projects in Awesome Lists tagged with inferentia
A curated list of projects in awesome lists tagged with inferentia .
https://github.com/vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
amd cuda gpt hpu inference inferentia llama llm llm-serving llmops mlops model-serving pytorch rocm tpu trainium transformer xpu
Last synced: 22 Dec 2024
https://github.com/pygmalionai/aphrodite-engine
Large-scale LLM inference engine
api-rest cuda inference-engine inferentia intel lora machine-learning rocm speculative-decoding tpu
Last synced: 18 Dec 2024
https://github.com/PygmalionAI/aphrodite-engine
Large-scale LLM inference engine
api-rest cuda inference-engine inferentia intel lora machine-learning rocm speculative-decoding tpu
Last synced: 03 Nov 2024
https://github.com/aws-samples/foundation-model-benchmarking-tool
Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
bedrock benchmark benchmarking evaluation-metrics foundation-models g5 g6 g6e generative-ai inferentia llama2 llama3 p4d p5 sagemaker trainium
Last synced: 21 Dec 2024
