https://github.com/DefTruth/Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
https://github.com/DefTruth/Awesome-LLM-Inference

List: awesome-llm-inference

awesome-llm deepseek flash-attention flash-attention-2 flash-attention-3 llm llm-inference llms open-sora paged-attention sora tensorrt-llm vllm

Last synced: about 1 month ago
JSON representation

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

• Host: GitHub
• URL: https://github.com/DefTruth/Awesome-LLM-Inference
• Owner: DefTruth
• License: gpl-3.0
• Created: 2023-08-27T02:32:15.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-10-28T02:38:17.000Z (7 months ago)
• Last Synced: 2024-10-30T03:44:54.310Z (7 months ago)
• Topics: awesome-llm, deepseek, flash-attention, flash-attention-2, flash-attention-3, llm, llm-inference, llms, open-sora, paged-attention, sora, tensorrt-llm, vllm
• Homepage: https://github.com/DefTruth/Awesome-LLM-Inference
• Size: 115 MB
• Stars: 2,738
• Watchers: 91
• Forks: 185
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-llm - LLM推理论文合集 - 以推理为主题的LLM相关论文汇总。 (其他相关论文)
• awesome-llm - LLM推理论文合集 - 以推理为主题的LLM相关论文汇总。 (其他相关论文)
• StarryDivineSky - DefTruth/Awesome-LLM-Inference - LLM、vLLM、streaming-llm、AWQ、SmoothQuant、WINT8/4、Continuous Batching、FlashAttention、PagedAttention 等。 (A01_文本生成_文本对话 / 大语言对话模型及数据)
• Awesome-LLM - Awesome-LLM-Inference - A curated list of Awesome LLM Inference Paper with codes. (Other Papers)
• awesome-ai-llm-development - Awesome LLM Inference
• awesome-ai-llm-development - Awesome LLM Inference
• awesome-awesome-artificial-intelligence - Awesome LLM Inference - LLM-Inference?style=social) | (Natural Language Processing)
• awesome-awesome-artificial-intelligence - Awesome LLM Inference - LLM-Inference?style=social) | (Natural Language Processing)
• awesome-llm-and-aigc - DefTruth/Awesome-LLM-Inference - LLM-Inference?style=social"/> : 📖A curated list of Awesome LLM/VLM Inference Papers with codes, such as FlashAttention, PagedAttention, Parallelism, etc. 🎉🎉 (Summary)
• awesome-ai-list-guide - Awesome-LLM-Inference - LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc. (NLP)
• ultimate-awesome - Awesome-LLM-Inference - 📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc. (Other Lists / Julia Lists)

README

        
# Notes 👇👇

This project has been moved to [xlite-dev/Awesome-LLM-Inference](https://github.com/xlite-dev/Awesome-LLM-Inference). Please check [xlite-dev/Awesome-LLM-Inference](https://github.com/xlite-dev/Awesome-LLM-Inference) for latest updates! 👏👋

---