https://github.com/DefTruth/Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
https://github.com/DefTruth/Awesome-LLM-Inference

List: Awesome-LLM-Inference

awesome-llm awq flash-attention flash-attention-2 flash-decoding inferflow kv-quant mamba paged-attention sora streaming-llm streamingllm tensorrt-llm vllm

Last synced: 3 months ago
JSON representation

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

• Host: GitHub
• URL: https://github.com/DefTruth/Awesome-LLM-Inference
• Owner: DefTruth
• License: gpl-3.0
• Created: 2023-08-27T02:32:15.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-03-15T03:32:31.000Z (3 months ago)
• Last Synced: 2024-03-15T04:42:51.360Z (3 months ago)
• Topics: awesome-llm, awq, flash-attention, flash-attention-2, flash-decoding, inferflow, kv-quant, mamba, paged-attention, sora, streaming-llm, streamingllm, tensorrt-llm, vllm
• Homepage: https://github.com/DefTruth/Awesome-LLM-Inference
• Size: 114 MB
• Stars: 946
• Watchers: 41
• Forks: 59
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Lists

• Awesome-LLM - Awesome-LLM-Inference - A curated list of Awesome LLM Inference Paper with codes. (Other Papers)
• Awesome-LLM?tab=readme-ov-file - Awesome-LLM-Inference - A curated list of Awesome LLM Inference Paper with codes. (Other Papers)
• awesome-llm-and-aigc - DefTruth/Awesome-LLM-Inference - LLM-Inference?style=social"/> : 📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc. (Summary)
• ultimate-awesome - Awesome-LLM-Inference - 📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc. (Other Lists / Julia Lists)
• StarryDivineSky - DefTruth/Awesome-LLM-Inference - LLM、vLLM、streaming-llm、AWQ、SmoothQuant、WINT8/4、Continuous Batching、FlashAttention、PagedAttention 等。 (文本生成、文本对话 / 类ChatGPT大语言对话模型及数据)